Search NASA⌕ Search

SEARCH · Search NASA

Results for “large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,243 records · Page 69

MO-A-BRD-09: A Data-Mining Algorithm for Large Scale Analysis of Dose-Outcome Relationships in a Database of Irradiated Head-And-Neck (HN) Cancer Patients

Purpose: To develop a framework for automatic extraction of clinically meaningful dosimetric-outcome relationships from an in-house, analytic oncology database. Methods: Dose-volume histograms (DVH) and clinical outcome-related structured data elements have been routinely stored to our database for 513 HN cancer patients treated from 2007 to 2014. SQL queries were developed to extract outcomes that had been assessed for at least 100 patients, as well as DVH curves for organs-at-risk (OAR) that were contoured for at least 100 patients. DVH curves for paired OAR (e.g., left and right parotids) were automatically combined and included as additional structures for analysis. For each OAR-outcome combination, DVH dose points, D(V{sub t}), at a series of normalized volume thresholds, V{sub t}=[0.01,0.99], were stratified into two groups based on outcomes after treatment completion. The probability, P[D(V{sub t})], of an outcome was modeled at each V{sub t} by logistic regression. Notable combinations, defined as having P[D(V{sub t})] increase by at least 5% per Gy (p<0.05), were further evaluated for clinical relevance using a custom graphical interface. Results: A total of 57 individual and combined structures and 115 outcomes were queried, resulting in over 6,500 combinations for analysis. Of these, 528 combinations met the 5%/Gy requirement, with further manual inspection revealing a number of reasonable models based on either reported literature or proximity between neighboring OAR. The data mining algorithm confirmed the following well-known toxicity/outcome relationships: dysphagia/larynx, voice changes/larynx, esophagitis/esophagus, xerostomia/combined parotids, and mucositis/oral mucosa. Other notable relationships included dysphagia/pharyngeal constrictors, nausea/brainstem, nausea/spinal cord, weight-loss/mandible, and weight-loss/combined parotids. Conclusion: Our database platform has enabled large-scale analysis of dose-outcome relationships. The current data-mining framework revealed both known and novel dosimetric and clinical relationships, underscoring the potential utility of this analytic approach. Multivariate models may be necessary to further evaluate the complex relationship between neighboring OARs and observed outcomes. This research was supported through collaborations with Elekta, Philips, and Toshiba.

60 APPLIED LIFE SCIENCES↗

Reducing the Analytical Bottleneck for Domain Scientists: Lessons from a Climate Data Visualization Case Study

The gap between large-scale data production rate and the rate of generation of data-driven scientific insights has led to an analytical bottleneck in scientific domains like climate, biology, etc. This is primarily due to the lack of innovative analytical tools that can help scientists efficiently analyze and explore alternative hypotheses about the data, and communicate their findings effectively to a broad audience. In this paper, by reflecting on a set of successful collaborative research efforts between with a group of climate scientists and visualization researchers, we introspect how interactive visualization can help reduce the analytical bottleneck for domain scientists.

climate↗

Predicting ptychography probe positions using single-shot phase retrieval neural network

Ptychography is a powerful imaging technique that is used in a variety of fields, including materials science, biology, and nanotechnology. However, the accuracy of the reconstructed ptychography image is highly dependent on the accuracy of the recorded probe positions which often contain errors. These errors are typically corrected jointly with phase retrieval through numerical optimization approaches. When the error accumulates along the scan path or when the error magnitude is large, these approaches may not converge with satisfactory result. We propose a fundamentally new approach for ptychography probe position prediction for data with large position errors, where a neural network is used to make single-shot phase retrieval on individual diffraction patterns, yielding the object image at each scan point. The pairwise offsets among these images are then found using a robust image registration method, and the results are combined to yield the complete scan path by constructing and solving a linear equation. We show that our method can achieve good position prediction accuracy for data with large and accumulating errors on the order of 10 2 pixels, a magnitude that often makes optimization-based algorithms fail to converge. For ptychography instruments without sophisticated position control equipment such as interferometers, our method is of significant practical potential.

47 OTHER INSTRUMENTATION↗

Cloud Giovanni: Reining in Costs and Improving Performance with Analytical Data Stores Using Scalable Serverless Architecture

Giovanni is the Geospatial Interactive Online Visualization ANd aNalysis Infrastructure developed at NASA GES DISC which provides a simple and intuitive way to visualize, analyze, and access vast amounts of Earth science data. It receives large number of user requests each day for a variety of analysis and visualization services, which leads to the big data challenge of serving gradually increasing large data volumes with diverse statistical algorithms. We hereby propose a multi-dimensional accumulation method which provides fast and cost-efficient cloud analysis for diverse services including both area averaging and time averaging. This method involves the weighted volume integration over multiple variable dimensions (time and space), and is implemented in AWS using Athena providing serverless and highly scalable data analysis. Compared to the standard method, this approach dramatically reduced the computational time by order of magnitude with a minimal AWS cost incurred. For example, for a benchmark of 10-year area averaging over the 1x1 degree daily variable, the computational time was reduced from minutes to seconds, and the Athena cost is only $5 for 100,000 requests.

Zhang, Hailiang↗

Neural Network for Analysis of Hyperspectral Imagery

Each material has unique spectral distribution that can be viewed as its fingerprint. Advanced hyper and ultra spectral sensors will produce large amount of data. Therefore, advanced capabilities are required that can effectively deal with large data cubes, possibly in real time.

hyperspectral imagery neural network↗

Capturing Historic Reliability Performance Through Graph Databases: A Model Based System Engineering Approach

With the goal of improving the performance and reliability of high dependable technological systems such as nuclear power plants, advanced monitoring and health management systems are employed to inform system engineers on observed degradation processes and anomalous behaviors of assets and components. This information is captured in the form of large amount of data which can be heterogenous in nature (e.g., numeric, textual). Such large data availability poses challenges when system engineers are required to parse and analyze them in order to track historic reliability performance of assets and components. This paper tackles directly this challenge by providing means to organize data in the form of a graph: a knowledge graph. The presented approach distinguish itself from current knowledge graph-based methods by the fact that model-based system engineering (MBSE) models are used to “put data into context”. In particular, MBSE models are used as skeleton of a knowledge graph; numeric and textual data elements, once processed, are associated to MBSE model elements. Thus, a knowledge graph captures both system architecture (though MBSE models) and health/performance data. Such feature opens the door to new data analytics methods designed to identify causal relations between observed phenomena.

97 - MATHEMATICS AND COMPUTING↗

MELT POOL MONITORING FOR CONTROL AND DATA ANALYTICS IN LARGE-SCALE METAL ADDITIVE MANUFACTURING

Laser-wire based Directed Energy Deposition (DED) processes are being developed at Oak Ridge National Laboratory’s Manufacturing Demonstration Facility, in collaboration with GKN Aerospace, for large volume metal additive manufacturing of metallic structures. The technology has the potential to deliver reduced costs and lead times when compared to conventional methods of manufacturing in several industries. While complex structures are being produced with relatively high deposition rates and at near-net shape, issues persist with achieving consistent geometric accuracy and thermal stability. A significant research effort is focused on developing coordinated, multi-modal sensing and control for addressing these issues. An introduction to this large-volume metal process will be provided, followed by investigations into real-time thermal monitoring capabilities that support control and data analytics frameworks. Studies focused on melt pool monitoring via infrared thermography, thermal and geometric effects on melt pool size measurement, and interactions between process variables and thermal monitoring metrics are presented.

Gibson, Brian↗

Enabling Diverse Software Stacks on Supercomputers using High Performance Virtual Clusters.

While large-scale simulations have been the hallmark of the High Performance Computing (HPC) community for decades, Large Scale Data Analytics (LSDA) workloads are gaining attention within the scientific community not only as a processing component to large HPC simulations, but also as standalone scientific tools for knowledge discovery. With the path towards Exascale, new HPC runtime systems are also emerging in a way that differs from classical distributed com- puting models. However, system software for such capabilities on the latest extreme-scale DOE supercomputing needs to be enhanced to more appropriately support these types of emerging soft- ware ecosystems. In this paper, we propose the use of Virtual Clusters on advanced supercomputing resources to enable systems to support not only HPC workloads, but also emerging big data stacks. Specifi- cally, we have deployed the KVM hypervisor within Cray's Compute Node Linux on a XC-series supercomputer testbed. We also use libvirt and QEMU to manage and provision VMs directly on compute nodes, leveraging Ethernet-over-Aries network emulation. To our knowledge, this is the first known use of KVM on a true MPP supercomputer. We investigate the overhead our solution using HPC benchmarks, both evaluating single-node performance as well as weak scaling of a 32-node virtual cluster. Overall, we find single node performance of our solution using KVM on a Cray is very efficient with near-native performance. However overhead increases by up to 20% as virtual cluster size increases, due to limitations of the Ethernet-over-Aries bridged network. Furthermore, we deploy Apache Spark with large data analysis workloads in a Virtual Cluster, ef- fectively demonstrating how diverse software ecosystems can be supported by High Performance Virtual Clusters.

97 MATHEMATICS AND COMPUTING↗

Email-Based Informed Consent: Innovative Method for Reaching Large Numbers of Subjects for Data Mining Research

Since the 2010 NASA authorization to make the Life Sciences Data Archive (LSDA) and Lifetime Surveillance of Astronaut Health (LSAH) data archives more accessible by the research and operational communities, demand for data has greatly increased. Correspondingly, both the number and scope of requests have increased, from 142 requests fulfilled in 2011 to 224 in 2014, and with some datasets comprising up to 1 million data points. To meet the demand, the LSAH and LSDA Repositories project was launched, which allows active and retired astronauts to authorize full, partial, or no access to their data for research without individual, study-specific informed consent. A one-on-one personal informed consent briefing is required to fully communicate the implications of the several tiers of consent. Due to the need for personal contact to conduct Repositories consent meetings, the rate of consenting has not kept up with demand for individualized, possibly attributable data. As a result, other methods had to be implemented to allow the release of large datasets, such as release of only de-identified data. However the compilation of large, de-identified data sets places a significant resource burden on LSAH and LSDA and may result in diminished scientific usefulness of the dataset. As a result, LSAH and LSDA worked with the JSC Institutional Review Board Chair, Astronaut Office physicians, and NASA Office of General Counsel personnel to develop a "Remote Consenting" process for retrospective data mining studies. This is particularly useful since the majority of the astronaut cohort is retired from the agency and living outside the Houston area. Originally planned as a method to send informed consent briefing slides and consent forms only by mail, Remote Consenting has evolved into a means to accept crewmember decisions on individual studies via their method of choice: email or paper copy by mail. To date, 100 emails have been sent to request participation in eight HRP-funded studies. The development of the Remote Consent process, the laws allowing transmission of consent via electronic means, total metrics to date, and remaining challenges (e.g., response issues, use of International Partner data, biospecimens/genetic data) for the research use of LSAH/LSDA data will be described.

Lee, Lesley R.↗

Choosing the best partition of the output from a large-scale simulation

Data partitioning becomes necessary when a large-scale simulation produces more data than can be feasibly stored. The goal is to partition the data, typically so that every element belongs to one and only one partition, and store summary information about the partition, either a representative value plus an estimate of the error or a distribution. Once the partitions are determined and the summary information stored, the raw data is discarded. This process can be performed in-situ; meaning while the simulation is running. When creating the partitions there are many decisions that researchers must make. For instance, how to determine once an adequate number of partitions have been created, how are the partitions created with respect to dividing the data, or how many variables should be considered simultaneously. In addition, decisions must be made for how to summarize the information within each partition. Because of the combinatorial number of possible ways to partition and summarize the data, a method of comparing the different possibilities will help guide researchers into choosing a good partitioning and summarization scheme for their application.

97 MATHEMATICS AND COMPUTING↗

Previously Unrecognized Large Lunar Impact Basins Revealed by Topographic Data

The discovery of a large population of apparently buried impact craters on Mars, revealed as Quasi- Circular Depressions (QCDs) in Mars Orbiting Laser Altimeter (MOLA) data [1,2,3] and as Circular Thin Areas (CTAs) [4] in crustal thickness model data [5] leads to the obvious question: are there unrecognized impact features on the Moon and other bodies in the solar system? Early analysis of Clementine topography revealed several large impact basins not previously known [6,7], so the answer certainly is "Yes." How large a population of previously undetected impact basins, their size frequency distribution, and how much these added craters and basins will change ideas about the early cratering history and Late Heavy Bombardment on the Moon remains to be determined. Lunar Orbiter Laser Altimeter (LOLA) data [8] will be able to address these issues. As a prelude, we searched the state-of-the-art global topographic grid for the Moon, the Unified Lunar Control Net (ULCN) [9] for evidence of large impact features not previously recognized by photogeologic mapping, as summarized by Wilhelms [lo].

Frey, Herbert V.↗

A Single Column Model Ensemble Approach Applied to the TWP-ICE Experiment

Single column models (SCM) are useful testbeds for investigating the parameterisation schemes of numerical weather prediction and climate models. The usefulness of SCM simulations are limited, however, by the accuracy of the best-estimate large-scale data prescribed. One method to address this uncertainty is to perform ensemble simulations of the SCM. This study first derives an ensemble of large-scale data for the Tropical Warm Pool International Cloud Experiment (TWP-ICE) based on an estimate of a possible source of error in the best-estimate product. This data is then used to carry out simulations with 11 SCM and 2 cloud-resolving models (CRM). Best-estimate simulations are also performed. All models show that moisture related variables are close to observations and there are limited differences between the best-estimate and ensemble mean values. The models, however, show different sensitivities to changes in the forcing particularly when weakly forced. The ensemble simulations highlight important differences in the moisture budget between the SCM and CRM. Systematic differences are also apparent in the ensemble mean vertical structure of cloud variables. The ensemble is further used to investigate relations between cloud variables and precipitation identifying large differences between CRM and SCM. This study highlights that additional information can be gained by performing ensemble simulations enhancing the information derived from models using the more traditional single best-estimate simulation.

applied↗

Data Compression Algorithm Architecture for Large Depth-of-Field Particle Image Velocimeters

A large depth-of-field particle image velocimeter (PIV) is designed to characterize dynamic dust environments on planetary surfaces. This instrument detects lofted dust particles, and senses the number of particles per unit volume, measuring their sizes, velocities (both speed and direction), and shape factors when the particles are large. To measure these particle characteristics in-flight, the instrument gathers two-dimensional image data at a high frame rate, typically >4,000 Hz, generating large amounts of data for every second of operation, approximately 6 GB/s. To characterize a planetary dust environment that is dynamic, the instrument would have to operate for at least several minutes during an observation period, easily producing more than a terabyte of data per observation. Given current technology, this amount of data would be very difficult to store onboard a spacecraft, and downlink to Earth. Since 2007, innovators have been developing an autonomous image analysis algorithm architecture for the PIV instrument to greatly reduce the amount of data that it has to store and downlink. The algorithm analyzes PIV images and automatically reduces the image information down to only the particle measurement data that is of interest, reducing the amount of data that is handled by more than 10(exp 3). The state of development for this innovation is now fairly mature, with a functional algorithm architecture, along with several key pieces of algorithm logic, that has been proven through field test data acquired with a proof-of-concept PIV instrument.

Bos, Brent↗

Volcanism on Venus as inferred from the morphometry of large shields

Morphometric data for the Venusian shields are compared with those for large volcanos on Mars and the Big Island shield in Hawaii. Reasonable assumptions and constraints on the chemical and physical properties of venusian magma, crust, and lithosphere are used with the morphometric data to estimate probable deep magma-source depths below the shields. Such data for selected large volcanos on Venus, Mars, and earth are presented. The flattened profiles of the Venusian shields are interpreted to be predominantly the result of the density contrast between lava and rocks between the source and surface, a relatively shallow depth to a primary magma source, low magma viscosity, high magma volumes and rates, and the inferred presence of flanking vents and lava tubes that carried lavas great distances from the summit. The measured shield heights are used in conjunction with a reasonable range of magma and rock densities to estimate the depths to deep magma sources, yielding an average of 25 km beneath the Venusian plains with a range depth of 3 to 40 km.

Schaber, G. G.↗

Using observational mean-flow data to drive large-eddy simulations of a diurnal cycle at the SWiFT site

Reproducing realistic date- and site-specific unsteady wind conditions in large-eddy simulations is becoming increasingly useful in wind energy. How to run a large-eddy simulation to match observed conditions, however, remains an open research question. One approach that has received considerable attention is mesoscale-to-microscale coupling, in which information about the mesoscale weather, most commonly acquired from a mesoscale numerical weather model, is passed on to a microscale model. In this paper, we demonstrate how the recently developed profile-assimilation technique, a form of mesoscale-to-microscale coupling, can be used to drive large-eddy simulations solely based on observed mean-flow profiles at a single location, bypassing the need for auxiliary mesoscale simulations. The new approach is evaluated for a diurnal cycle at the Scaled Wind Farm Technology site. Observed mean-flow profiles from the ground up to a height of 2 km are reconstructed by aggregating measurements from multiple instruments, and gaps in the data are infilled with natural neighbor interpolation. We perform nine simulations using various forcing approaches to deal with data limitations. The results show that it is indeed possible to drive microscale large-eddy simulation with observations using the profile-assimilation technique, notwithstanding large gaps in virtual potential temperature measurements. However, profile assimilation with vertical smoothing of the error between the desired and actual profiles is required. Without that smoothing, the microscale simulations develop unrealistically high turbulence levels under many situations. Finally, we show that simulated mesoscale data can account for missing observations, although care is needed as both data sources are not necessarily compatible.

17 WIND ENERGY↗

Trajectory Segmentation Map-Matching Approach for Large-Scale, High-Resolution GPS Data

With the development of smartphones and portable GPS devices, large-scale, high-resolution GPS data can be collected. Map matching is a critical step in studying vehicle driving activity and recognizing network traffic conditions from the data. A new trajectory segmentation map-matching algorithm is proposed to deal accurately and efficiently with large-scale, high-resolution GPS trajectory data. The new algorithm separated the GPS trajectory into segments. It found the shortest path for each segment in a scientific manner and ultimately generated a best-matched path for the entire trajectory. The similarity of a trajectory segment and its matched path is described by a similarity score system based on the longest common subsequence. The numerical experiment indicated that the proposed map-matching algorithm was very promising in relation to accuracy and computational efficiency. Large-scale data set applications verified that the proposed method is robust and capable of dealing with real-world, large-scale GPS data in a computationally efficient and accurate manner.

97 MATHEMATICS AND COMPUTING↗