Search NASA⌕ Search

SEARCH · Search NASA

Results for “streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,477 records · Page 82

F3D Image Processing and Analysis for Many - and Multi-core Platforms

F3D is written in OpenCL, so it achieve[sic] platform-portable parallelism on modern mutli-core CPUs and many-core GPUs. The interface and mechanims to access F3D core are written in Java as a plugin for Fiji/ImageJ to deliver several key image-processing algorithms necessary to remove artifacts from micro-tomography data. The algorithms consist of data parallel aware filters that can efficiently utilizes[sic] resources and can work on out of core datasets and scale efficiently across multiple accelerators. Optimizing for data parallel filters, streaming out of core datasets, and efficient resource and memory and data managements over complex execution sequence of filters greatly expedites any scientific workflow with image processing requirements. F3D performs several different types of 3D image processing operations, such as non-linear filtering using bilateral filtering and/or median filtering and/or morphological operators (MM). F3D gray-level MM operators are one-pass constant time methods that can perform morphological transformations with a line-structuring element oriented in discrete directions. Additionally, MM operators can be applied to gray-scale images, and consist of two parts: (a) a reference shape or structuring element, which is translated over the image, and (b) a mechanism, or operation, that defines the comparisons to be performed between the image and the structuring element. This tool provides a critical component within many complex pipelines such as those for performing automated segmentation of image stacks. F3D is also called a "descendent" of Quant-CT, another software we developed in the past. These two modules are to be integrated in a next version. Further details were reported in: D.M. Ushizima, T. Perciano, H. Krishnan, B. Loring, H. Bale, D. Parkinson, and J. Sethian. Structure recognition from high-resolution images of ceramic composites. IEEE International Conference on Big Data, October 2014.

Source record↗

Ice Flow in the North East Greenland Ice Stream

Early observations with ERS-1 SAR image data revealed a large ice stream in North East Greenland (Fahnestock 1993). The ice stream has a number of the characteristics of the more closely studied ice streams in Antarctica, including its large size and gross geometry. The onset of rapid flow close to the ice divide and the evolution of its flow pattern, however, make this ice stream unique. These features can be seen in the balance velocities for the ice stream (Joughin 1997) and its outlets. The ice stream is identifiable for more than 700 km, making it much longer than any other flow feature in Greenland. Our research goals are to gain a greater understanding of the ice flow in the northeast Greenland ice stream and its outlet glaciers in order to assess their impact on the past, present, and future mass balance of the ice sheet. We will accomplish these goals using a combination of remotely sensed data and ice sheet models. We are using satellite radar interferometry data to produce a complete maps of velocity and topography over the entire ice stream. We are in the process of developing methods to use these data in conjunction with existing ice sheet models similar to those that have been used to improve understanding of the mechanics of flow in Antarctic ice streams.

Joughin, Ian↗

HPC ODA Commons [SWR-26-003]

HPC ODA Commons is a community-driven platform for standardizing HPC operational data analytics. HPC sites generate enormous volumes of operational data - scheduler logs, accounting records, monitoring streams - but turning that data into actionable insight is needlessly hard. Each site builds bespoke parsers, schemas, and evaluation pipelines. Results can't be compared across institutions. Promising analytics ideas stay siloed because there's no shared language for describing the data, the experiments, or the outcomes. HPC ODA Commons fixes this by establishing community-governed contracts - versioned schemas, canonical artifacts, and benchmark recipes - that make ODA workflows discoverable, reproducible, and comparable. It pairs these standards with a practical, CLI-first toolkit that lets operators and researchers go from raw logs to standardized results without sending data off-cluster.

Menear, Kevin [National Laboratory of the Rockies ↗

Experimental investigation of shock-cell noise reduction for single-stream nozzles in simulated flight, comprehensive data report. Volume 2: Laser velocimeter data

Mean velocity (axial component) and turbulent velocity (axial component) measurements for thirty one selected flow conditions of six models were performed employing the Laser Doppler Velocimeter Aerodynamic conditions which define the test points are given. Tabulations which explain the scope of mean velocity traverses and turbulence histogram measurements are also presented. The actual LV position, the type of traverse, and measured mean and turbulent velocities along copies of the LV mean velocity traces are contained.

Yamamoto, K.↗

Experimental investigation of shock-cell noise reduction for single-stream nozzles in simulated flight, comprehensive data report. Volume 3: Shadowgraph photos and facility description

A total of 142 shadowgraph photographs were taken on 43 different plumes that were distributed over the six nozzle configurations using the 9.5 inch diameter collimated light beam of the shadowgraph setup. Aerodynamic flow conditions of the shadowgraph test points, the location and identification of each of the photographs, and copies of the pictures are presented.

Yamamoto, K.↗

The crush of new earth science data knocking at our door

The reasons for collecting massive amounts of earth science data in the Earth Observing System (EOS) Project are discussed. A processing hierarchy for handling the data is described, and the prospects for adequate throughput and storage for operational data analysis in the EOS era are addressed. Needs for successful exploratory data analysis are examined, and the policy issues implicated by the large stream of EOS data are considered.

Kahn, Ralph↗

Real-time forecasting of electricity demand in a streams-based architecture with applications

A streams platform is used. Multiple streams of electricity usage data are received, each from an electrical meter providing periodic updates to electrical usage for devices connected to the electrical meter. Weather information is received corresponding to locations where the electrical meters are. Real-time predictive modeling of electricity demand is performed based on the received multiple streams of electricity usage data and the received weather information, at least by performing: updating a state space model for electrical load curves using the usage data from the streams and the weather, wherein the updating uses current load observations for the multiple streams for a current time period; and creating forecast(s) for the electricity demand. The forecast(s) of the electricity demand are output. Appliance-level predictions may be made and used, and substitution effects and load management functions may be performed.

Ghosh, Soumyadip↗

The Pristine survey – XVI. The metallicity of 26 stellar streams around the Milky Way detected with the STREAMFINDER in Gaia EDR3

ABSTRACT We use the photometric metallicities provided by the panoramic Pristine survey to study the veracity and derive the metallicities of the numerous stellar streams found by the application of the STREAMFINDER algorithm to the Gaia Early Data Release 3 data. All 26 streams present in Pristine show a clear metallicity distribution function, which provides an independent check of the reality of these structures, supporting the reliability of STREAMFINDER in finding streams and the power of Pristine to measure precise metallicities. We further present six candidate structures with coherent phase-space and metallicity signals that are very likely streams. The majority of studied streams are very metal-poor (14 structures with [Fe/H] < −2.0) and include three systems with [Fe/H] < −2.9 (C-11, C-19, and C-20). These streams could be the closest debris of low-luminosity dwarf galaxies or may have originated from globular clusters of significantly lower metallicity than any known current Milky Way globular cluster. Our study shows that the promise of the Gaia data for Galactic Archeology studies can be substantially strengthened by quality photometric metallicities, allowing us to peer back into the earliest epochs of the formation of our Galaxy and its stellar halo constituents.

79 ASTRONOMY AND ASTROPHYSICS↗

Using integrated data analysis to extend measurement capability (invited)

The analysis approach called integrated data analysis (IDA) provides a means to exploit all information present in multiple streams of raw data to produce the best inference of a plasma parameter. This contrasts with the typical approach in which information (data) from a single diagnostic is used to measure a given parameter, e.g., visible bremsstrahlung → Z eff . Data from a given diagnostic usually contain information on many parameters. For example, a Thomson scattering diagnostic is sensitive to bremsstrahlung and line emission in addition to electron temperature. This background light is typically subtracted off and discarded but could be used to improve knowledge of Z eff . IDA encourages explicit awareness of such information and provides the quantitative framework to exploit it. This gives IDA the ability to increase spatial and temporal resolution, increase precision and accuracy of inferences, and measure plasma parameters that are difficult or impossible to measure using single diagnostic techniques. One example is the measurement of Z eff on Madison symmetric torus using IDA since no single diagnostic can provide a robust measurement. Thus, as we enter the burning plasma era, application of IDA will be critical to the measurement of certain parameters, as diagnostic access in the harsh fusion environment will be extremely limited.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

File Specification for GEOS-5 FP (Forward Processing)

The GEOS-5 FP Atmospheric Data Assimilation System (GEOS-5 ADAS) uses an analysis developed jointly with NOAA's National Centers for Environmental Prediction (NCEP), which allows the Global Modeling and Assimilation Office (GMAO) to take advantage of the developments at NCEP and the Joint Center for Satellite Data Assimilation (JCSDA). The GEOS-5 AGCM uses the finite-volume dynamics (Lin, 2004) integrated with various physics packages (e.g, Bacmeister et al., 2006), under the Earth System Modeling Framework (ESMF) including the Catchment Land Surface Model (CLSM) (e.g., Koster et al., 2000). The GSI analysis is a three-dimensional variational (3DVar) analysis applied in grid-point space to facilitate the implementation of anisotropic, inhomogeneous covariances (e.g., Wu et al., 2002; Derber et al., 2003). The GSI implementation for GEOS-5 FP incorporates a set of recursive filters that produce approximately Gaussian smoothing kernels and isotropic correlation functions. The GEOS-5 ADAS is documented in Rienecker et al. (2008). More recent updates to the model are presented in Molod et al. (2011). The GEOS-5 system actively assimilates roughly 2 × 10(exp 6) observations for each analysis, including about 7.5 × 10(exp 5) AIRS radiance data. The input stream is roughly twice this volume, but because of the large volume, the data are thinned commensurate with the analysis grid to reduce the computational burden. Data are also rejected from the analysis through quality control procedures designed to detect, for example, the presence of cloud. To minimize the spurious periodic perturbations of the analysis, GEOS-5 FP uses the Incremental Analysis Update (IAU) technique developed by Bloom et al. (1996). More details of this procedure are given in Appendix A. The assimilation is performed at a horizontal resolution of 0.3125-degree longitude by 0.25- degree latitude and at 72 levels, extending to 0.01 hPa. All products are generated at the native resolution of the horizontal grid. The majority of data products are time-averaged, but four instantaneous products are also available. Hourly data intervals are used for two-dimensional products, while 3-hourly intervals are used for three-dimensional products. These may be on the model's native 72-layer vertical grid or at 42 pressure surfaces extending to 0.1 hPa. This document describes the gridded output files produced by the GMAO near real-time operational FP, using the most recent version of the GEOS-5 assimilation system. Additional details about variables listed in this file specification can be found in a separate document, the GEOS-5 File Specification Variable Definition Glossary. Documentation about the current access methods for products described in this document can be found on the GMAO products page: http://gmao.gsfc.nasa.gov/products/.

GSI↗

File Specification for GEOS-5 FP-IT (Forward Processing for Instrument Teams)

The GEOS-5 FP-IT Atmospheric Data Assimilation System (GEOS-5 ADAS) uses an analysis developed jointly with NOAA's National Centers for Environmental Prediction (NCEP), which allows the Global Modeling and Assimilation Office (GMAO) to take advantage of the developments at NCEP and the Joint Center for Satellite Data Assimilation (JCSDA). The GEOS-5 AGCM uses the finite-volume dynamics (Lin, 2004) integrated with various physics packages (e.g, Bacmeister et al., 2006), under the Earth System Modeling Framework (ESMF) including the Catchment Land Surface Model (CLSM) (e.g., Koster et al., 2000). The GSI analysis is a three-dimensional variational (3DVar) analysis applied in grid-point space to facilitate the implementation of anisotropic, inhomogeneous covariances (e.g., Wu et al., 2002; Derber et al., 2003). The GSI implementation for GEOS-5 FP-IT incorporates a set of recursive filters that produce approximately Gaussian smoothing kernels and isotropic correlation functions. The GEOS-5 ADAS is documented in Rienecker et al. (2008). More recent updates to the model are presented in Molod et al. (2011). The GEOS-5 system actively assimilates roughly 2 × 10(exp 6) observations for each analysis, including about 7.5 × 10(exp 5) AIRS radiance data. The input stream is roughly twice this volume, but because of the large volume, the data are thinned commensurate with the analysis grid to reduce the computational burden. Data are also rejected from the analysis through quality control procedures designed to detect, for example, the presence of cloud. To minimize the spurious periodic perturbations of the analysis, GEOS-5 FP-IT uses the Incremental Analysis Update (IAU) technique developed by Bloom et al. (1996). More details of this procedure are given in Appendix A. The analysis is performed at a horizontal resolution of 0.625-degree longitude by 0.5-degree latitude and at 72 levels, extending to 0.01 hPa. All products are generated at the native resolution of the horizontal grid. The majority of data products are time-averaged, but four instantaneous products are also available. Hourly data intervals are used for two-dimensional products, while 3-hourly intervals are used for three-dimensional products. These may be on the model's native 72-layer vertical grid or at 42 pressure surfaces extending to 0.1 hPa. This document describes the gridded output files produced by the GMAO near real-time operational GEOS-5 FP-IT processing in support of the EOS instrument teams. Additional details about variables listed in this file specification can be found in a separate document, the GEOS-5 File Specification Variable Definition Glossary.

3D Var↗

Data Archive and Portal (DAP) Platform for Solid Phase Processing Technologies

The scale and speed of data generated by modern scientific experiments have constantly challenged the research community to store, curate, manage and optimally use it to drive scientific discoveries. In this work, we have developed a data archive and portal (DAP) platform including analytics capabilities to collect, curate, and manage data and metadata stream for solid phase processing (SPP) techniques. We successfully hosted around ~347K files of data related to processing parameters, microscopic images, and spectroscopic data related to solid phase processing. The DAP platform for SPP will establish an enduring capability to support machine learning and grow collaboration at the intersection of materials science and data science.

36 MATERIALS SCIENCE↗