Search NASA⌕ Search

SEARCH · Search NASA

Results for “Large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

UNIX-based data management system for the Mobile Satellite Propagation Experiment (PiFEx)

A new method is presented for handling data resulting from Mobile Satellite propagation experiments such as the Pilot Field Experiment (PiFEx) conducted by JPL. This method uses the UNIX operating system and C programming language. The data management system is implemented on a VAX minicomputer. The system automatically divides the large data file housing data from various experiments under a predetermined format into various individual files containing data from each experiment. The system also has a number of programs written in C and FORTRAN languages to allow the researcher to obtain meaningful quantities from the data at hand.

Kantak, Anil V.↗

Accessing and visualizing scientific spatiotemporal data

This paper discusses work done by JPL's Parallel Applications Technologies Group in helping scientists access and visualize very large data sets through the use of multiple computing resources, such as parallel supercomputers, clusters, and grids.

rendering↗

Sensor Management for Applied Research Technologies (SMART)-On Demand Modeling (ODM) Project

NASA requires timely on-demand data and analysis capabilities to enable practical benefits of Earth science observations. However, a significant challenge exists in accessing and integrating data from multiple sensors or platforms to address Earth science problems because of the large data volumes, varying sensor scan characteristics, unique orbital coverage, and the steep learning curve associated with each sensor and data type. The development of sensor web capabilities to autonomously process these data streams (whether real-time or archived) provides an opportunity to overcome these obstacles and facilitate the integration and synthesis of Earth science data and weather model output. A three year project, entitled Sensor Management for Applied Research Technologies (SMART) - On Demand Modeling (ODM), will develop and demonstrate the readiness of Open Geospatial Consortium (OGC) Sensor Web Enablement (SWE) capabilities that integrate both Earth observations and forecast model output into new data acquisition and assimilation strategies. The advancement of SWE-enabled systems (i.e., use of SensorML, sensor planning services - SPS, sensor observation services - SOS, sensor alert services - SAS and common observation model protocols) will have practical and efficient uses in the Earth science community for enhanced data set generation, real-time data assimilation with operational applications, and for autonomous sensor tasking for unique data collection.

Goodman, M.↗

The Sensor Management for Applied Research Technologies (SMART) Project

NASA seeks on-demand data processing and analysis of Earth science observations to facilitate timely decision-making that can lead to the realization of the practical benefits of satellite instruments, airborne and surface remote sensing systems. However, a significant challenge exists in accessing and integrating data from multiple sensors or platforms to address Earth science problems because of the large data volumes, varying sensor scan characteristics, unique orbital coverage, and the steep "learning curve" associated with each sensor, data type, and associated products. The development of sensor web capabilities to autonomously process these data streams (whether real-time or archived) provides an opportunity to overcome these obstacles and facilitate the integration and synthesis of Earth science data and weather model output.

Goodman, Michael↗

Evolving HPC and Application Design Toward a Coupled Data Assimilation System at NASA Suitable for Emerging Exascale Platforms

The prediction capabilities of global models have continuously evolved from the traditional medium-range global weather prediction application to span scales in support of hourly prediction of convective scale storms to seasonal Earth system prediction. This evolution has increased the demands on the system infrastructure design and workflow to achieve the required performance on modern high-performance computing (HPC) platforms. The planned evolution of the Goddard Earth Observing System (GEOS) modeling and assimilation system will stress the capabilities of conventional HPC overwhelming the available compute cycles at the NASA Center for Climate Simulation (NCCS) at the NASA Goddard Space Flight Center in the coming 5-10 years. This has led to the re-design of key elements of the assimilation and modeling systems to achieve significant gains in performance on anticipated Exacale platforms. The transition of the assimilation system to the Joint Effort for Data assimilation Integration (JEDI) framework has positioned GEOS to exploit new efficient algorithms for data assimilation (DA) in a fully-coupled Earth system context. The suitability of the GEOS model to leverage a domain specific language (DSL) approach and artificial intelligence (AI) is being explored to accelerate computational performance and data exchange efficiency of the coupled Earth system model. The storage and processing of large data volumes produced by these advance systems is being redesigned with a data-centric cloud-based approach. We will highlight the recent efforts in these areas and emphasize the demand for further development and re-design to achieve the science objectives in support of NASA's Earth system modeling and assimilation missions.

Putman, Bill↗

Next Generation Big Data Storage for Long Space Missions

This paper presents the results of the HELIOS (Hardened Extremely Long Life In-formation Optical Storage) mission on the International Space Station (ISS) which tested a unique solution for the long-term storage and retrieval of data in space. For this mission Creative Technology (CTech) developed test media—termed WORF (Write Once, Read Forever)—to validate whether this patented technology will survive all critical parameters for harsh space-based environments including microgravity and ionizing radiation. The HELIOS experiment confirmed that the WORF media is impervious to ionizing radiation, microgravity, solar (plasma) eruptions, and the stress from 8 Gs of the launch including extreme temperature expo-sure. The principal results indicate that there has been no discernible degradation of the media after 8 months on the ISS as compared to a control set of media stored on the ground. This data validated the media’s survivability for harsh space environments for long-term and deep space missions. In addition to the space environment, we are confident that WORF technology can be used for data storage where space-related and other long-term or archival integrity is critical such as: geospatial collections from satellites; space weather archives; past, ongoing, and future space mission media and documentation files; the deep space Gate-way program; as well as Big Data applications such as the Vera C. Rubin astronomical observatory (formerly the LSST). WORF technology for the HELIOS experiment uses a proven archival media, redesigned, re-purposed and patented by CTech to store digital data for long periods, measured in decades and possibly centuries. The media stores standing waves embedded in a substrate that capture the precise col-ors or wavelengths projected onto the media. The colors represent numerical data, with each data location storing multiple superimposed wavelengths, which facilitate the storage of multiple data bytes (rather than just zeros and ones); advanced mathematical permutations allow for extremely large data density equal to or greater than contemporary data storage de-vices. These colors cannot fade or degrade over time since the standing waves are physically stabilized (fully oxidized) metallic silver; no dyes are embedded for this storage system, and silver ions resist micro-bacterial and fungal contamination

Rodney Grubbs↗

Improving the Automatic Inversion of Digital ISIS-2 Ionogram Reflection Traces into Topside Vertical Electron-Density Profiles

The topside-sounders on the four satellites of the International Satellites for Ionospheric Studies (ISIS) program were designed as analog systems. The resulting ionograms were displayed on 35-mm film for analysis by visual inspection. Each of these satellites, launched between 1962 and 1971, produced data for 10 to 20 years. A number of the original telemetry tapes from this large data set have been converted directly into digital records. Software, known as the TOPside Ionogram Scalar with True-height (TOPIST) algorithm has been produced that enables the automatic inversion of ISIS-2 ionogram reflection traces into topside vertical electron-density profiles Ne(h). More than million digital Alouette/ISIS topside ionograms have been produced and over 300,000 are from ISIS 2. Many of these ISIS-2 ionograms correspond to a passive mode of operation for the detection of natural radio emissions and thus do not contain ionospheric reflection traces. TOPIST, however, is not able to produce Ne(h) profiles from all of the ISIS-2 ionograms with reflection traces because some of them did not contain frequency information. This information was missing due to difficulties encountered during the analog-to-digital conversion process in the detection of the ionogram frame-sync pulse and/or the frequency markers. Of the many digital topside ionograms that TOPIST was able to process, over 200 were found where direct comparisons could be made with Ne(h) profiles that were produced by manual scaling in the early days of the ISIS program. While many of these comparisons indicated excellent agreement (<10% average difference over the entire profile) there were also many cases with large differences (more than a factor of two). Here we will report on two approaches to improve the automatic inversion process: (1) improve the quality of the digital ionogram database by remedying the missing frequency-information problem when possible, and (2) using the above-mentioned comparisons as teaching examples of how to improve the original TOPIST software.

Benson, R. F.↗

Solving differential equations using deep neural networks

Recent work on solving partial differential equations (PDEs) with deep neural networks (DNNs) is presented. The paper reviews and extends some of these methods while carefully analyzing a fundamental feature in numerical PDEs and nonlinear analysis: irregular solutions. First, the Sod shock tube solution to the compressible Euler equations is discussed and analyzed. This analysis includes a comparison of a DNN-based approach with conventional finite element and finite volume methods, and demonstrates that the DNN is competitive in terms of degrees of freedom required for a given accuracy. Further, the DNN-based approach is extended to consider performance improvements and simultaneous parameter space exploration. Next, a shock solution to compressible magnetohydrodynamics (MHD) is solved for, and used in a scenario where experimental data is utilized to enhance a PDE system that is a priori insufficient to validate against the observed/experimental data. This is accomplished by enriching the model PDE system with source terms that are then inferred via supervised training with synthetic experimental data. The resulting DNN framework for PDEs enables straightforward system prototyping and natural integration of large data sets (be they synthetic or experimental), all while simultaneously enabling single-pass exploration of an entire parameter space.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

QUOTAS: A New Research Platform for the Data-driven Discovery of Black Holes

We present QUOTAS, a novel research platform for the data-driven investigation of supermassive black hole (SMBH) populations. While SMBH data—observations and simulations—have grown in complexity and abundance, our computational environments and tools have not matured commensurately to exhaust opportunities for discovery. To explore the BH, host galaxy, and parent dark matter halo connection—in this pilot version—we assemble and colocate the high-redshift, z > 3 quasar population alongside simulated data at the same cosmic epochs. As a first demonstration of the utility of QUOTAS, we investigate correlations between observed Sloan Digital Sky Survey (SDSS) quasars and their hosts with those derived from simulations. Leveraging machine-learning algorithms (ML), to expand simulation volumes, we show that halo properties extracted from smaller dark-matter-only simulation boxes successfully replicate halo populations in larger boxes. Next, using the Illustris-TNG300 simulation that includes baryonic physics as the training set, we populate the larger LEGACY Expanse dark-matter-only box with quasars, and show that observed SDSS quasar occupation statistics are accurately replicated. First science results from QUOTAS comparing colocated observational and ML-trained simulated data at z3 are presented. QUOTAS demonstrates the power of ML, in analyzing and exploring large data sets, while also offering a unique opportunity to interrogate theoretical assumptions that underpin accretion and feedback models. QUOTAS and all related materials are publicly available at the Google Kaggle platform. (The full data set—observational data and simulation data—are available at: https://www.kaggle.com/ and the codes are available at:https://www.kaggle.com/datasets/quotasplatform/quotas)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

An interdisciplinary analysis of multispectral satellite data for selected cover types in the Colorado Mountains, using automatic data processing techniques

The author has reported the following significant results. A data set containing SKYLAB, LANDSAT, and topographic data has been overlayed, registered, and geometrically corrected to a scale of 1:24,000. After geometrically correcting both sets of data, the SKYLAB data were overlayed on the LANDSAT data. Digital topographic data were then obtained, reformatted, and a data channel containing elevation information was then digitally overlayed onto the LANDSAT and SKYLAB spectral data. The 14,039 square kilometers involving 2,113, 776 LANDSAT pixels represents a relatively large data set available for digital analysis. The overlayed data set enables investigators to numerically analyze and compare two sources of spectral data and topographic data from any point in the scene. This capability is new and it will permit a numerical comparison of spectral response with elevation, slope, and aspect. Utilization of the spectral and topographic data together to obtain more accurate classifications of the various cover types present is feasible.

Hoffer, R. M.↗

User's guide: Programs for processing altimeter data over inland seas

The programs described were developed to process GEODYN-formatted satellite altimeter data, and to apply the processed results to predict geoid undulations and gravity anomalies of inland sea areas. These programs are written in standard FORTRAN 77 and are designed to run on the NSESCC IBM 3081(MVS) computer. Because of the experimental nature of these programs they are tailored to the geographical area analyzed. The attached program listings are customized for processing the altimeter data over the Black Sea. Users interested in the Caspian Sea data are expected to modify each program, although the required modifications are generally minor. Program control parameters are defined in the programs via PARAMETER statements and/or DATA statements. Other auxiliary parameters, such as labels, are hard-wired into the programs. Large data files are read in or written out through different input or output units. The program listings of these programs are accompanied by sample IBM job control language (JCL) images. Familiarity with IBM JCL and the TEMPLATE graphic package is assumed.

Au, A. Y.↗

Upper tropospheric water vapor features observed in the 1980's using the 6.7 and 7.2 micron TOVS channels

The NOAA has been preprocessing the TIROS Operational Vertical Sounder (TOVS) IR and microwave channels to correct for cloud amount, spot size, and viewing angle. An examination is now being conducted of the 6-7 micron water vapor band; a preliminary evaluation is here presented of the regional and global features of this large data base on daily, monthly, annual, and decadal time scales. The TOVS archives allow large-scale climatological features to be established.

Chesters, Dennis↗

Eye Safe, Visible Wavelength Lidar Systems: Design and Operational Advances, Results and Potential

In the early nineties the first of the eye safe visible wavelength lidar systems known now as Micro Pulse Lidar (MPL) became operational. The important advance of the design was a system that, unlike most existing lidar, operated at eye safe energy densities and could thus operate unattended for full time monitoring. Since that time there have been many dozens of these systems produced and applied for full time profiling of atmospheric cloud and aerosol structure. There is currently an observational network of MPL sites to support global climate research. In thc course of application of these instruments there have been significant improvements in the, design and performance of the systems. In the last half decade particularly there has been significant application and technical development of MPL systems. In this paper we review progress. The current MPL systems in use are all single wavelength systems designed for cloud and aerosol applications. For the cloud and aerosol applications, both lidar depolarization and multi wavelength measurements have significant applications. These can be accomplished with the MPL, approach. The main current challenge for the lidar network activity are in the area of the reliability, repeatability and efficiency of data processing. The network makes use of internet data downloads and automated processing. The heights of all cloud and aerosol layers are needed. The recent emphasis has been in operationally deriving aerosol extinction cross section. Future emphasis will include adding cirrus optical parameters. For operational effectiveness, improvements to simplify routine data signal calibration are being researched. Overall the MPL systems have proven very effective. A large data base of results from globally distributed sites can be easily accessed through the internet. Applications have included atmospheric model development. Validation of current global satellite observations of aerosol and clouds, including now orbital lidar observations, was a primary goal for NASA. Although sampling issues require careful consideration, results have proven useful.

Spinhirne, James↗

Development of RGB Composite Imagery for Operational Weather Forecasting Applications

The NASA Short-term Prediction Research and Transition (SPoRT) Center, in collaboration with the Cooperative Institute for Research in the Atmosphere (CIRA), is providing red-green-blue (RGB) color composite imagery to several of NOAA s National Centers and National Weather Service forecast offices as a demonstration of future capabilities of the Advanced Baseline Imager (ABI) to be implemented aboard GOES-R. Forecasters rely upon geostationary satellite imagery to monitor conditions over their regions of responsibility. Since the ABI will provide nearly three times as many channels as the current GOES imager, the volume of data available for analysis will increase. RGB composite imagery can aid in the compression of large data volumes by combining information from multiple channels or paired channel differences into single products that communicate more information than provided by a single channel image. A standard suite of RGB imagery has been developed by the European Organization for the Exploitation of Meteorological Satellites (EUMETSAT), based upon the Spinning Enhanced Visible and Infrared Imager (SEVIRI). The SEVIRI instrument currently provides visible and infrared wavelengths comparable to the future GOES-R ABI. In addition, the Moderate Resolution Imaging Spectroradiometer (MODIS) instruments aboard the NASA Terra and Aqua satellites can be used to demonstrate future capabilities of GOES-R. This presentation will demonstrate an overview of the products currently disseminated to SPoRT partners within the GOES-R Proving Ground, and other National Weather Service forecast offices, along with examples of their application. For example, CIRA has used the channels of the current GOES sounder to produce an "air mass" RGB originally designed for SEVIRI. This provides hourly imagery over CONUS for looping applications while demonstrating capabilities similar to the future ABI instrument. SPoRT has developed similar "air mass" RGB imagery from MODIS, and through a case study example, synoptic-scale features evident in single-channel water vapor imagery are shown in the context of the air mass product. Other products, such as the "nighttime microphysics" RGB, are useful in the detection of low clouds and fog. Nighttime microphysics products from MODIS offer some advantages over single-channel or spectral difference techniques and will be discussed in the context of a case study. Finally, other RGB products from SEVIRI are being demonstrated as precursors to GOES-R within the GOES-R Proving Ground. Examples of "natural color" and "dust" imagery will be shown with relevant applications.

Molthan, Andrew L.↗

Improving the Automatic Inversion of Digital Alouette/ISIS Ionogram Reflection Traces into Topside Electron Density Profiles

The topside sounders of the International Satellites for Ionospheric Studies (ISIS) program were designed as analog systems. The resulting ionograms were displayed on 35 mm film for analysis by visual inspection. Each of these satellites, launched between 1962 and 1971, produced data for 10 to 20 years. A number of the original telemetry tapes from this large data set have been converted directly into digital records. Software, known as the Topside Ionogram Scalar With True-Height (TOPIST) algorithm, has been produced and used for the automatic inversion of the ionogram reflection traces on more than 100,000 ISIS-2 digital topside ionograms into topside vertical electron density profiles Ne(h). Here we present some topside ionospheric solar cycle variations deduced from the TOPIST database to illustrate the scientific benefit of improving and expanding the topside ionospheric Ne(h) database. The profile improvements will be based on improvements in the TOPIST software motivated by direct comparisons between TOPIST profiles and profiles produced by manual scaling in the early days of the ISIS program. The database expansion will be based on new software designed to overcome limitations in the original digital topside ionogram database caused by difficulties encountered during the analog-to-digital conversion process in the detection of the ionogram frame sync pulse and/or the frequency markers. This improved and expanded TOPIST topside Ne(h) database will greatly enhance investigations into both short- and long-term ionospheric changes, e.g., the observed topside ionospheric responses to magnetic storms, induced by interplanetary magnetic clouds, and solar cycle variations, respectively.

Benson, Robert F.↗

Machine Learning Prediction of Tritium‐Helium Groundwater Ages in the Central Valley, California, USA

Abstract Groundwater ages provides insight into recharge rates, flow velocities, and vulnerability to contaminants. The ability to predict groundwater ages based on more accessible parameters via Machine Learning (ML) would advance our ability to guide sustainable management of groundwater resources. In this study, ML models were trained and tested on a large data set of tritium concentrations and tritium‐helium groundwater ages from the California Central Valley, a large groundwater basin with complex land use, irrigation, and water management practices. The ML models were trained on 63 features, including location, well construction information, landscape characteristics, and climate variables, water chemistry, and stable isotopes. The Bagging regressor method can accurately classify (F1‐score = 0.91) groundwater samples as either modern or pre‐modern whereas the accuracy of the ML prediction of continuous tritium‐helium groundwater ages is limited and explains only of the variability in this data set. In general, ML groundwater age prediction relies mostly on features related to (a) the source of groundwater recharge, (b) contaminant history, (c) aquifer materials, (d) well construction, and (e) geochemical reactions along flow paths.

54 ENVIRONMENTAL SCIENCES↗

J-PLUS: Support vector regression to measure stellar parameters

Stellar parameters are among the most important characteristics in studies of stars which, in traditional methods, are based on atmosphere models. However, time, cost, and brightness limits restrain the efficiency of spectral observations. The Javalambre Photometric Local Universe Survey (J-PLUS) is an observational campaign that aims to obtain photometry in 12 bands. Owing to its characteristics, J-PLUS data have become a valuable resource for studies of stars. Machine learning provides powerful tools for efficiently analyzing large data sets, such as the one from J-PLUS, and enables us to expand the research domain to stellar parameters. The main goal of this study is to construct a support vector regression (SVR) algorithm to estimate stellar parameters of the stars in the first data release of the J-PLUS observational campaign. The training data for the parameter's regressions are featured with 12-waveband photometry from J-PLUS and are crossidentified with spectrum-based catalogs. These catalogs are from the Large Sky Area Multi-Object Fiber Spectroscopic Telescope, the Apache Point Observatory Galactic Evolution Experiment, and the Sloan Extension for Galactic Understanding and Exploration. We then label them with the stellar effective temperature, the surface gravity, and the metallicity. Ten percent of the sample is held out to apply a blind test. We develop a new method, a multi-model approach, in order to fully take into account, the uncertainties of both the magnitudes and the stellar parameters. The method utilizes more than 200 models to apply the uncertainty analysis. We present a catalog of 2 493 424 stars with the root mean square error of 160 K in the effective temperature regression, 0.35 in the surface gravity regression, and 0.25 in the metallicity regression. We also discuss the advantages of this multi-model approach and compare it to other machine-learning methods.

79 ASTRONOMY AND ASTROPHYSICS↗

A conceptual framework for using Doppler radar acquired atmospheric data for flight simulation

A concept is presented which can permit turbulence simulation in the vicinity of microbursts. The method involves a large data base, but should be fast enough for use with flight simulators. The model permits any pilot to simulate any flight maneuver in any aircraft. The model simulates a wind field with three-component mean winds and three-component turbulent gusts, and gust variation over the body of an aircraft so that all aerodynamic loads and moments can be calculated. The time and space variation of mean winds and turbulent intensities associated with a particular atmospheric phenomenon such as a microburst is used in the model. In fact, Doppler radar data such as provided by JAWS is uniquely suited for use with the proposed model. The concept is completely general and is not restricted to microburst studies. Reentry and flight in terrestrial or planetary atmospheres could be realistically simulated if supporting data of sufficient resolution were available.

Campbell, W.↗