Search NASA⌕ Search

SEARCH · Search NASA

Results for “Large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
NASA NTRS2012

JPL Genesis and Rapid Intensification Processes (GRIP) Portal

Satellite observations can play a very important role in airborne field campaigns, since they provide a comprehensive description of the environment that is essential for the experiment design, flight planning, and post-experiment scientific data analysis. In the past, it has been difficult to fully utilize data from multiple NASA satellites due to the large data volume, the complexity of accessing NASA s data in near-real-time (NRT), as well as the lack of software tools to interact with multi-sensor information. The JPL GRIP Portal is a Web portal that serves a comprehensive set of NRT observation data sets from NASA and NOAA satellites describing the atmospheric and oceanic environments related to the genesis and intensification of the tropical storms in the North Atlantic Ocean. Together with the model forecast data from four major global atmospheric models, this portal provides a useful tool for the scientists and forecasters in planning and monitoring the NASA GRIP field campaign during the 2010 Atlantic Ocean hurricane season. This portal uses the Google Earth plug-in to visualize various types of data sets, such as 2D maps, wind vectors, streamlines, 3D data sets presented at series of vertical cross-sections or pointwise vertical profiles, and hurricane best tracks and forecast tracks. Additionally, it allows users to overlap multiple data sets, change the opacity of each image layer, generate animations on the fly with selected data sets, and compare the observation data with the model forecast using two independent calendars. The portal also provides the capability to identify the geographic location of any point of interest. In addition to supporting the airborne mission planning, the NRT data and portal will serve as a very rich source of information during the post-field campaign analysis stage of the airborne experiment. By including a diverse set of satellite observations and model forecasts, it provides a good spatial and temporal context for the high-resolution, but limited in space and time, airborne observations.

Knosp, Brian W.↗
NASA NTRS1998

Predicting Ecologically Important Vegetation Variables from Remotely Sensed Optical/Radar Data Using Neural Networks

A number of satellite sensor systems will collect large data sets of the Earth's surface during NASA's Earth Observing System (EOS) era. Efforts are being made to develop efficient algorithms that can incorporate a wide variety of spectral data and ancillary data in order to extract vegetation variables required for global and regional studies of ecosystem processes, biosphere-atmosphere interactions, and carbon dynamics. These variables are, for the most part, continuous (e.g. biomass, leaf area index, fraction of vegetation cover, vegetation height, vegetation age, spectral albedo, absorbed photosynthetic active radiation, photosynthetic efficiency, etc.) and estimates may be made using remotely sensed data (e.g. nadir and directional optical wavelengths, multifrequency radar backscatter) and any other readily available ancillary data (e.g., topography, sun angle, ground data, etc.). Using these types of data, neural networks can: 1) provide accurate initial models for extracting vegetation variables when an adequate amount of data is available; 2) provide a performance standard for evaluating existing physically-based models; 3) invert multivariate, physically based models; 4) in a variable selection process, identify those independent variables which best infer the vegetation variable(s) of interest; and 5) incorporate new data sources that would be difficult or impossible to use with conventional techniques. In addition, neural networks employ a more powerful and adaptive nonlinear equation form as compared to traditional linear, index transformations, and simple nonlinear analyses. These neural networks attributes are discussed in the context of the authors' investigations of extracting vegetation variables of ecological interest.

Kimes, Daniel S.↗
NASA NTRS2005

Methods and apparatus for extraction and tracking of objects from multi-dimensional sequence data

An object tracking technique is provided which, given: (i) a potentially large data set; (ii) a set of dimensions along which the data has been ordered; and (iii) a set of functions for measuring the similarity between data elements, a set of objects are produced. Each of these objects is defined by a list of data elements. Each of the data elements on this list contains the probability that the data element is part of the object. The method produces these lists via an adaptive, knowledge-based search function which directs the search for high-probability data elements. This serves to reduce the number of data element combinations evaluated while preserving the most flexibility in defining the associations of data elements which comprise an object.

Hill, Matthew L.↗
NASA NTRS2008

Methods and apparatus for extraction and tracking of objects from multi-dimensional sequence data

An object tracking technique is provided which, given: (i) a potentially large data set; (ii) a set of dimensions along which the data has been ordered; and (iii) a set of functions for measuring the similarity between data elements, a set of objects are produced. Each of these objects is defined by a list of data elements. Each of the data elements on this list contains the probability that the data element is part of the object. The method produces these lists via an adaptive, knowledge-based search function which directs the search for high-probability data elements. This serves to reduce the number of data element combinations evaluated while preserving the most flexibility in defining the associations of data elements which comprise an object.

Hill, Matthew L.↗
NASA NTRS1993

LANL High-Performance Data System (HPDS)

The Los Alamos High-Performance Data System (HPDS) is being developed to meet the very large data storage and data handling requirements of a high-performance computing environment. The HPDS will consist of fast, large-capacity storage devices that are directly connected to a high-speed network and managed by software distributed in workstations. The HPDS model, the HPDS implementation approach, and experiences with a prototype disk array storage system are presented.

Collins, M. William↗
NASA NTRS1982

The use of large-area spectral data in wheat yield estimation

Large-area relations between satellite spectral data and end-of-season crop yield were investigated. Green Index Number (GIN) values from Landsat MSS data of sample segments throughout the U.S. Great Plains winter wheat belt in 1978 were correlated to county USDA-SRS reported yields. A linear relation between GIN and yield appeared to exist up to GIN values of 40 or 50, covering cases of severe to moderate stress. In a test on 1978 Texas winter wheat at the county level, GIN values for sample segments in the counties were used in conjunction with an agronomic-meteorological yield model. The combined fit explained significantly more of the observed yield variation at the county level than the agromet model alone.

Barnett, T. L.↗
DOE OSTI2024

Hybrid Storage Solution

With the rise of artificial intelligence and machine learning, data sets used to train models have become increasingly large. The availability, accessibility and integrity of large data sets has become important to the research conducted at Los Alamos National Laboratory. Ceph is a storage solution suitable for use with critical data because of its distributed nature and ability to keep multiple copies of a file in different locations. The amount of data means that bandwidth, latency, and cost are important factors and the reason most storage solutions are on-premises. However, there are distinct advantages to hosting services in the cloud, namely scalability and ease-of-use. In this paper, we explore the possibility of provisioning a hybrid Ceph cluster that leverages the benefits of both cloud architectures and on-premise performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A Comprehensive Characterization of Galaxy-cool CGM Connections at z < 0.4 with DESI Year 1 Data

We investigate the relationships between the cool circumgalactic medium (CGM), traced by Ca II absorption lines, and galaxy properties at z < 0.4 using ∼900,000 galaxy–quasar pairs within 200 kpc from the Year 1 data of the Dark Energy Spectroscopic Instrument (DESI). This large data set enables us to obtain composite spectra with sensitivity reaching to the mÅ level and to explore the Ca II absorption as a function of stellar mass, star formation rate (SFR), redshift, and galaxy types, including active galactic nuclei (AGNs). Our results show a positive correlation between the absorption strength and stellar mass of star-forming galaxies with $\langle$$W$$^{Ca II}_{0}$$\rangle$ α $M$$^{0.5}_{*}$ over 3 orders of magnitude in stellar mass from ∼10 8 to 10 11 M ⊙ , while such a mass dependence is weaker for quiescent galaxies. At a fixed mass, Ca II absorption is stronger around star-forming galaxies than quiescent ones especially within impact parameters <30 kpc. Among star-forming galaxies, the Ca II absorption further correlates with SFR, following ∝SFR 0.3 . However, in contrast to the results at higher redshifts, stronger absorption is not preferentially observed along the minor axis of star-forming galaxies, indicating a possible redshift evolution of CGM dynamics resulting from galactic feedback. Moreover, no significant difference between the properties of the cool gas around AGNs and galaxies is detected. Finally, we measure the absorption profiles with respect to the virial radius of dark matter halos and show that the total Ca II mass in the CGM is comparable to the Ca mass in the ISM of galaxies.

Circumgalactic medium↗
NASA NTRS1973

Digital processing of radiographic images

Some techniques are presented and the software documentation for the digital enhancement of radiographs. Both image handling and image processing operations are considered. The image handling operations dealt with are: (1) conversion of format of data from packed to unpacked and vice versa; (2) automatic extraction of image data arrays; (3) transposition and 90 deg rotations of large data arrays; (4) translation of data arrays for registration; and (5) reduction of the dimensions of data arrays by integral factors. Both the frequency and the spatial domain approaches are presented for the design and implementation of the image processing operation. It is shown that spatial domain recursive implementation of filters is much faster than nonrecursive implementations using fast fourier transforms (FFT) for the cases of interest in this work. The recursive implementation of a class of matched filters for enhancing image signal to noise ratio is described. Test patterns are used to illustrate the filtering operations. The application of the techniques to radiographic images of metallic structures is demonstrated through several examples.

Bond, A. D.↗
NASA NTRS1981

Generic functional requirements for a NASA general-purpose data base management system

Generic functional requirements for a general-purpose, multi-mission data base management system (DBMS) for application to remotely sensed scientific data bases are detailed. The motivation for utilizing DBMS technology in this environment is explained. The major requirements include: (1) a DBMS for scientific observational data; (2) a multi-mission capability; (3) user-friendly; (4) extensive and integrated information about data; (5) robust languages for defining data structures and formats; (6) scientific data types and structures; (7) flexible physical access mechanisms; (8) ways of representing spatial relationships; (9) a high level nonprocedural interactive query and data manipulation language; (10) data base maintenance utilities; (11) high rate input/output and large data volume storage; and adaptability to a distributed data base and/or data base machine configuration. Detailed functions are specified in a top-down hierarchic fashion. Implementation, performance, and support requirements are also given.

Lohman, G. M.↗
NASA NTRS1982

Pilot study of the domestic information display system in state and local government

An interactive computer based system that can retrieve a wide range of data (demographic, environmental, socio-economic, etc.,) from a large data base and display these data for different geographic units in the form of choropleth maps was developed. The system was designed to display statistical information in a geographic format for national policy makers.

Source record↗
NASA NTRS1985

Imaging spectrometry for earth remote sensing

Initial results of the novel remote earth sensing technique of imaging spectrometry, which is technically feasible from both spacecraft and aircraft platforms, indicate that the direct identification of surface materials on a picture-element basis is possible through proper sampling of absorption features in the reflectance spectrum. Sensors of this type are able to acquire images simultaneously in 100-200 contiguous spectral bands. Computerized data reduction and storage techniques are available for the large data sets thus generated, and novel analytic techniques are under development to maximize information content extraction.

Goetz, A. F. H.↗
NASA NTRS1987

SAR image data compression for an on-line archive system

This paper summarizes the investigation of SAR image data compression for an on-line archive data distribution system. This system is planned for the ground processing system of Alaska SAR Facility (ASF) and Shuttle Imaging Radar (SIR-C). The objective of the SAR image data compression is to enable the data archive system to provide the remote users a large data base with good image quality, short response time, low transfer cost, and minimal decoding complexity. The requirements and limitations of the on-line archive data distribution system are presented. The effects of SAR image data characteristics on data compression are addressed. The users' survey results suggest that compression ratios between 10:1 and 20:1 appear suitable. Based on the algorithm evaluation results, the two-level tree-searched vector quantization technique has been recommended as the SAR image data compression algorithm for the on-line archive data distribution system.

Chang, C. Y.↗
NASA NTRS2011

Online Time Series Analysis of Land Products over Asia Monsoon Region via Giovanni

Time series analysis is critical to the study of land cover/land use changes and climate. Time series studies at local-to-regional scales require higher spatial resolution, such as 1km or less, data. MODIS land products of 250m to 1km resolution enable such studies. However, such MODIS land data files are distributed in 10ox10o tiles, due to large data volumes. Conducting a time series study requires downloading all tiles that include the study area for the time period of interest, and mosaicking the tiles spatially. This can be an extremely time-consuming process. In support of the Monsoon Asia Integrated Regional Study (MAIRS) program, NASA GES DISC (Goddard Earth Sciences Data and Information Services Center) has processed MODIS land products at 1 km resolution over the Asia monsoon region (0o-60oN, 60o-150oE) with a common data structure and format. The processed data have been integrated into the Giovanni system (Goddard Interactive Online Visualization ANd aNalysis Infrastructure) that enables users to explore, analyze, and download data over an area and time period of interest easily. Currently, the following regional MODIS land products are available in Giovanni: 8-day 1km land surface temperature and active fire, monthly 1km vegetation index, and yearly 0.05o, 500m land cover types. More data will be added in the near future. By combining atmospheric and oceanic data products in the Giovanni system, it is possible to do further analyses of environmental and climate changes associated with the land, ocean, and atmosphere. This presentation demonstrates exploring land products in the Giovanni system with sample case scenarios.

Shen, Suhung↗
NASA NTRS2009

Onboard Classification of Hyperspectral Data on the Earth Observing One Mission

Remote-sensed hyperspectral data represents significant challenges in downlink due to its large data volumes. This paper describes a research program designed to process hyperspectral data products onboard spacecraft to (a) reduce data downlink volumes and (b) decrease latency to provide key data products (often by enabling use of lower data rate communications systems). We describe efforts to develop onboard processing to study volcanoes, floods, and cryosphere, using the Hyperion hyperspectral imager and onboard processing for the Earth Observing One (EO-1) mission as well as preliminary work targeting the Hyperspectral Infrared Imager (HyspIRI) mission.

cryosphere↗
NASA NTRS2018

Advanced Analytics and Big Earth Data

NASA's Earth Science Data Systems process, archive and distribute petabytes of Earth Observation data to a variety of end users. These end users will face dramatically increased data size in the near future, bringing about new challenges and opportunities in analyzing those data. One area of particular ferment currently is Machine Learning. Many Machine Learning methods are black boxes, limiting direct insight into the data's properties. However, they can be used for a variety of data enhancement purposes, such as parameter retrieval, data fusion and image classification and segmentation. The Earth Observing System Data and Information System is also evolving to host large data volumes in the cloud, enabling data proximal analysis. As part of this effort, an Analytics framework is being developed to support and enhance user analysis of the data. By using standards based services in the framework, diverse user communities can be served, while also allowing inter-system collaboration in the analysis process.

Cloud Computing↗
NASA NTRS2024

SMAnalytics Automated Shape Memory Test Analysis Software User's Manual: UCFTC, UPER, and DSC Test Methods, Version 1.0.2

Over the course of 60 years of shape memory alloy research and development, the properties of these alloys have been measured using various testing methods, which are often customized by the organization performing the test. However, commercial adoption of shape memory alloys in aeronautic actuator applications requires reducing property uncertainty through standardizing test methods. Historically, differential scanning calorimetry (DSC) has been used to measure transformation temperatures under stress-free conditions. This testing method is well established, and tests are run under the applicable standard, ASTM F2004-17: Standard Test Method for Transformation Temperature of Nickel-Titanium Alloys by Thermal Analysis. Two other test methods for measuring strains and transformation temperatures under constant load or free recovery after prestraining have only recently been standardized—ASTM E3097: Standard Test Method for Mechanical Uniaxial Constant Force Thermal Cycling of SMAs (UCFTC) and ASTM E3098: Standard Test Method for Mechanical Uniaxial Pre-Strain and Thermal Free Recovery of SMAs (UPFR). These standards represent a critical step forward in producing reliable material property data for the use of these alloys in aeronautics and other commercial areas. However, no standard programs or software packages were previously available for postprocessing UCFTC- and UPFR-type test data to extract property data. Each organization that performed the testing analyzed data using its own custom manual methods, such as using a plot and a straight edge, spreadsheet plotting methods, or custom code routines, to extract the required values for strains, stresses, and transformation temperatures. SMAnalytics represents the first publicly available uniform software package for processing data generated using the DSC, UCFTC, and UPFR test methods, as well as many modifications of these methods, including loading in martensite versus austenite and performing multiple thermal cycles at stress. In doing so, it provides a tool for reducing potential error or variability in the measured values due to the precision of the technique used and person-to-person variability. It is also expected that this automated data-parsing tool can accelerate the data analysis phase of the test campaign, especially for large data files. This User’s Manual describes how the SMAnalytics software works and the method for its use.

Shape Memory Alloy↗
DOE OSTI2022

Data Storage for HEP Experiments in the Era of High-Performance Computing

As particle physics experiments push their limits on both the energy and the intensity frontiers, the amount and complexity of the produced data are also expected to increase accordingly. With such large data volumes, next-generation efforts like the HL-LHC and DUNE will rely even more on both high-throughput (HTC) and high-performance (HPC) computing clusters. Full utilization of HPC resources requires scalable and efficient data-handling and I/O. For the last few decades, ROOT has been used by most HEP experiments to store data. However, other storage technologies like HDF5 may perform better in HPC environments. Initial explorations with HDF5 have begun using ATLAS, CMS and DUNE data; the DUNE experiment has also adopted HDF5 for its data-acquisition system. This paper presents the future outlook of the HEP computing and the role of HPC, and a summary of ongoing and future works to use HDF5 as a possible data storage technology for the HEP experiments to use in HPC environments.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗