Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Challenge”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 847 records · Page 47

Implementation of Dipolar Resonant Excitation Collision Induced Dissociation with Ion Mobility/Time-of-Flight MS

Under and overfragmentation are significant hurdles to the data independent “bottom-up” approach to proteomics. Another challenge to the data independent approach is the convolution of fragments from different peptides that coelute in reverse-phase liquid chromatography/mass spectrometry (RPLC/MS). The ion mobility/collision induced dissociation/time-of flight mass spectrometry (IMS/CID/TOF MS) approach gives drift-time aligned fragment ions that have the same arrival time distributions as precursor ions, greatly aiding in fragment and peptide ion identification. We have modified an IMS/TOF MS platform to allow for resonant excitation CID experiments. Resonant excitation CID leads to highly efficient, mass-resolved fragmentation without additional excitation of product ions, alleviating the overfragmentation problem. The ability to apply resonant waveforms in mobility-resolved windows has been demonstrated with a peptide mixture yielding fragmentation over a range of mass-to-charge (m/z) ratios within a single IMS separation experiment.

Environmental Molecular Sciences Laboratory↗

Daily quality assurance software for a satellite radiometer system

Six Differential Microwave Radiometers (DMR) on COBE (Cosmic Background Explorer) measure the large-angular-scale isotropy of the cosmic microwave background (CMB) at 31.5, 53, and 90 GHz. Quality assurance software analyzes the daily telemetry from the spacecraft to ensure that the instrument is operating correctly and that the data are not corrupted. Quality assurance for DMR poses challenging requirements. The data are differential, so a single bad point can affect a large region of the sky, yet the CMB isotropy requires lengthy integration times (greater than 1 year) to limit potential CMB anisotropies. Celestial sources (with the exception of the moon) are not, in general, visible in the raw differential data. A 'quicklook' software system was developed that, in addition to basic plotting and limit-checking, implements a collection of data tests as well as long-term trending. Some of the key capabilities include the following: (1) stability analysis showing how well the data RMS averages down with increased data; (2) a Fourier analysis and autocorrelation routine to plot the power spectrum and confirm the presence of the 3 mK 'cosmic' dipole signal; (3) binning of the data against basic spacecraft quantities such as orbit angle; (4) long-term trending; and (5) dipole fits to confirm the spacecraft attitude azimuth angle.

Keegstra, P. B.↗

Analyzing Martian winds and tracer concentrations using Mars Observer data

During the courses of a day, the Mars Observer spacecraft will acquire globally distributed profiles of the martian atmosphere. It is highly desirable that this data be assembled into synoptic weather maps (complete specifications of the atmospheric pressure, temperature, and winds at a given time), which can in turn be used as starting points in the study of many meteorological phenomena. Unfortunately, the special nature of the Mars Observer data presents several challenges above and beyond the usual difficult problem of data initialization. Mars Observer atmospheric data will consist almost exclusively of asynoptic vertical profiles of temperatures (or radiances) and pressures, whereas winds are generally in balance with horizontal gradients of these quantities (which will not be observed). It will therefore be necessary to resort to dynamical models to analyze the wind fields. As a rule, data assimilation into atmospheric models can result in the generation of spurious gravity waves, so special steps must be taken to suppress these. In addition, the asynoptic nature of the data will require a four-dimensional (space and time) data assimilation scheme. The problem is to find a full set of meteorological fields (winds and temperatures) such that, when marched forward in time in the model, they achieve a best fit (in the weighted least-squares sense) to the data. The proposed solution is to develop a model especially for the Mars Observer data assimilation problem. Gravity waves are filtered from the model by eliminating all divergence terms from the prognostic divergence equation. This leaves a diagnostic gradient wind relation between the rotational wind and the temperature field. The divergent wind is diagnosed as the wind required to maintain the gradient wind balance in the presence of the diabatic heating. The primitive equations of atmospheric dynamics (with three principal dependent variables) are thus reduced to a simpler system with a single prognostic equation for temperature - the variable that will be best observed. (This balance system was apparently first derived by Charney as a first-order Rossby number expansion of the equations of motion). Experience with a full primitive equation model of the martian atmosphere indicates that a further simplification is possible: at least for short-term integrations, the model can be linearized about the zonally symmetric basic state.

Houben, Howard C.↗

Revisiting Huffman Coding: Toward Extreme Performance on Modern GPU Architectures

Today's high-performance computing (HPC) applications are producing vast volumes of data, which are challenging to store and transfer efficiently during the execution, such that data compression is becoming a critical technique to mitigate the storage burden and data movement cost. Huffman coding is arguably the most efficient Entropy coding algorithm in information theory, such that it could be found as a fundamental step in many modern compression algorithms such as DEFLATE. On the other hand, today's HPC applications are more and more relying on the accelerators such as GPU on supercomputers, while Huffman encoding suffers from low throughput on GPUs, resulting in a significant bottleneck in the entire data processing. In this paper, we propose and implement an efficient Huffman encoding approach based on modern GPU architectures, which addresses two key challenges: (1) how to parallelize the entire Huffman encoding algorithm, including codebook construction, and (2) how to fully utilize the high memory-bandwidth feature of modern GPU architectures. The detailed contribution is fourfold. (1) We develop an efficient parallel codebook construction on GPUs that scales effectively with the number of input symbols. (2) We propose a novel reduction based encoding scheme that can efficiently merge the codewords on GPUs. (3) We optimize the overall GPU performance by leveraging the state-of-the-art CUDA APIs such as Cooperative Groups. (4) We evaluate our Huffman encoder thoroughly using six real-world application datasets on two advanced GPUs and compare with our implemented multi-threaded Huffman encoder. Experiments show that our solution can improve the encoding throughput by up to 5.0x and 6.8x on NVIDIA RTX 5000 and V100, respectively, over the state-of-the-art GPU Huffman encoder, and by up to 3.3x over the multi-thread encoder on two 28-core Xeon Platinum 8280 CPUs.

Tian, Jiannan↗

Revisiting Huffman Coding: Toward Extreme Performance on Modern GPU Architectures

Today's high-performance computing (HPC) applications are producing vast volumes of data, which are challenging to store and transfer efficiently during the execution, such that data compression is becoming a critical technique to mitigate the storage burden and data movement cost. Huffman coding is arguably the most efficient Entropy coding algorithm in information theory, such that it could be found as a fundamental step in many modern compression algorithms such as DEFLATE. On the other hand, today's HPC applications are more and more relying on the accelerators such as GPU on supercomputers, while Huffman encoding suffers from low throughput on GPUs, resulting in a significant bottleneck in the entire data processing. In this paper, we propose and implement an efficient Huffman encoding approach based on modern GPU architectures, which addresses two key challenges: (1) how to parallelize the entire Huffman encoding algorithm, including codebook construction, and (2) how to fully utilize the high memory-bandwidth feature of modern GPU architectures. The detailed contribution is fourfold. (1) We develop an efficient parallel codebook construction on GPUs that scales effectively with the number of input symbols. (2) We propose a novel reduction based encoding scheme that can efficiently merge the codewords on GPUs. (3) We optimize the overall GPU performance by leveraging the state-of-the-art CUDA APIs such as Cooperative Groups. (4) We evaluate our Huffman encoder thoroughly using six real-world application datasets on two advanced GPUs and compare with our implemented multithreaded Huffman encoder. Experiments show that our solution can improve the encoding throughput by up to 5.0× and 6.8× on NVIDIA RTX 5000 and V100, respectively, over the state-of-the-art GPU Huffman encoder, and by up to 3.3× over the multithread encoder on two 28-core Xeon Platinum 8280 CPUs.

Tian, Jiannan↗

Future Online Analysis Platform Workshop Report

Recent technology trends, such as the increasing focus on data management technologies and the emergence of sustainable on-demand computing and commercial cloud facilities provide initial steps and potential building blocks for creating such a compute facility. How do we fill the gap between what we have now and the capabilities we need to make the vision of an online data facility a reality? What research challenges need to be addressed – and can be addressed – in the coming 5-10 years? How should we adjust our vision if they are not solved? Is such a facility compatible with the existing and evolving exascale resources and/or how should they evolve to be compatible? How do “beyond Moore’s law” technologies enable advanced data analysis? Those are the challeges that the Future Platform workshop was organized to address. This report is organized as follows. Section 2 summarizes data challenges and patterns related to the exemplar use cases presented during the first half day of the workshop. Section 3 summarizes the requirements presented by those use cases as well as a synthesis of those requirements into a Future Platform definition achieved on the 3rd half day of the workshop (morning of the 2nd day). Section 4 includes area breakout reports from the areas on the 2nd half day of the workshop (afternoon of the 1st day). Finally Section 5 has a report on recommendation for building methodology and metrics. The report concludes in Section 6.

97 MATHEMATICS AND COMPUTING↗

A method for processing the Confidential Carload Waybill Sample for railroad freight analysis

Freight transportation research is often constrained by the availability of appropriate data. Within the context of freight rail research, use of traditional freight datasets present some critical challenges. Aggregating freight data to geographic areas diminishes the ability to perform nuanced analysis. More challenging, the complexity of railroad operations limits the usability of publicly available freight flow data which do not account for which rail company is economically and legally responsible for the movement of the goods or for the contractual arrangements between infrastructure owners and carriers. Finally, detailed data are usually privately owned and not readily-accessible for analysis outside the individual rail companies. As part of their regulatory authority, the Surface Transportation Board has developed the Confidential Carload Waybill Sample, a stratified sample representing 1% to 3% of all rail traffic in the United States for a particular year. While the sample is rich with data, it is limited in spatial information as well as in its implementation in geospatial analysis. This paper introduces a methodology for geospatially enabling the Confidential Carload Waybill Sample which can then be used to generate rail routes based on a topological representation of the U.S. railroad network. As a result, this methodology is applied using two case studies, a freight diversion analysis and transporting hazardous materials.

42 ENGINEERING↗

Active Time-Domain Reflectometry for Unattended Safeguards Systems FY15 Report

The International Atomic Energy Agency (IAEA) continues to expand its use of unattended measurement systems. An increasing number of systems and an expanding family of instruments create challenges in terms of deployment efficiency and the implementation of data authentication measures. In collaboration with the IAEA, tamper-indicating measures to address data-transmission authentication challenges with unattended safeguards systems are under investigation. Pacific Northwest National Laboratory (PNNL) is studying the viability of active time-domain reflectometry (TDR) along two parallel but interconnected paths: (1) swept-frequency TDR as the highly flexible, laboratory gold standard to which field-deployable options can be compared, and (2) a low-cost commercially available spread-spectrum TDR technology as one option for field implementation. This report describes PNNL’s FY15 progress in the viability study including: an overview of the TDR methods under investigation; description of the testing configurations and mock tampering scenarios; results from a preliminary sensitivity comparison of the two TDR methods; demonstration of a quantitative metric for estimating field performance that acknowledges the need for high detection probability while minimizing false alarms. FY15 progress reported here sets the stage for a rigorous comparison of the candidate TDR methods, over a range of deployment scenarios and perturbing effects typical of IAEA unattended monitoring systems.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Combined Statistical Analyses of Peptide Intensities and Peptide Occurrences Improves Identification of Significant Peptides from MS-based Proteomics Data

Liquid chromatography-mass spectrometry-based (LC-MS) proteomics uses peak intensities of proteolytic peptides to infer the differential abundance of peptides/proteins. However, substantial run-to-run variability in peptide intensities and observations (presence/absence) of peptides makes data analysis quite challenging. The missing abundance values in LC-MS proteomics data are difficult to address with traditional imputation-based approaches because the mechanisms by which data are missing are unknown a priori. Data can be missing due to random mechanisms such as experimental error, or non-random mechanisms such as a true biological effect. We present a statistical approach that uses a test of independence known as a G-test to test the null hypothesis of independence between the number of missing values and the experimental groups. We pair the G-test results evaluating independence of missing data (IMD) with a standard analysis of variance (ANOVA) that uses only means and variances computed from the observed data. Each peptide is therefore represented by two statistical confidence metrics, one for qualitative differential observation and one for quantitative differential intensity. We use two simulated and two real LC-MS datasets to demonstrate the robustness and sensitivity of the ANOVA-IMD approach for assigning confidence to peptides with significant differential abundance among experimental groups.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Conflation of Geospatial POI Data and Ground-level Imagery

The code solves the problem of conflating POI (Points of Interest) geospatial data and geo-tagged ground-level imagery data. The main challenges with fusion or conflation of these two types of data has been that these two data entities are represented not only in several ways in current state-of-the-art, but also there is mismatch of representation format of these two data types. This source code/software brings POI datapoints and ground-level imagery datapoints into same representation format, and then generates valuable Knowledge Graph combining POI and images data (so that it can be used for multitude of applications).

De, Debraj↗

Time-Domain Reflectometry for Tamper Indication in Unattended Monitoring Systems for Safeguards

The International Atomic Energy Agency (IAEA) continues to expand its use of unattended, remotely monitored measurement systems. An increasing number of systems and an expanding family of instruments create challenges in terms of deployment efficiency and the implementation of data authentication measures. Pacific Northwest National Laboratory (PNNL) leads a collaboration that is exploring various tamper-indicating (TI) measures that could help to address some of the long-standing detector and data-transmission authentication challenges with IAEA’s unattended systems. PNNL is investigating the viability of active time-domain reflectometry (TDR) along two parallel but interconnected paths: (1) swept-frequency TDR as the highly flexible, laboratory gold standard to which field-deployable options can be compared, and (2) a low-cost commercially available spread-spectrum TDR technology as one option for field implementation. This report describes PNNL’s progress and preliminary findings from the first year of the study, and describes the path forward.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

BIOME: A browser-aware search and order system

The Oak Ridge National Laboratory (ORNL) Distributed Active Archive Center (DAAC), which is associated with NASA's Earth Observing System Data and Information System (EOSDIS), provides access to a large number of tabular and imagery datasets used in ecological and environmental research. Because of its large and diverse data holdings, the challenge for the ORNL DAAC is to help users find data of interest from the hundreds of thousands of files available at the DAAC without overwhelming them. Therefore, the ORNL DAAC developed the Biogeochemical Information Ordering Management Environment (BIOME), a search and order system for the World Wide Web (WWW). The WWW provides a new vehicle that allows a wide range of users access to the data. This paper describes the specialized attributes incorporated into BIOME that allow researchers easy access to an otherwise bewildering array of data products.

Grubb, Jon W.↗

BIOME: A scientific data archive search-and-order system using browser-aware, dynamic pages

The Oak Ridge National Laboratory's (ORNL) Distributed Active Archive Center (DAAC) is a data archive and distribution center for the National Air and Space Administration's (NASA) Earth Observing System Data and Information System (EOSDIS). Both the Earth Observing System (EOS) and EOSDIS are components of NASA's contribution to the US Global Change Research Program through its Mission to Planet Earth Program. The ORNL DAAC provides access to data used in ecological and environmental research such as global change, global warming, and terrestrial ecology. Because of its large and diverse data holdings, the challenge for the ORNL DAAC is to help users find data of interest from the hundreds of thousands of files available at the DAAC without overwhelming them. Therefore, the ORNL DAAC has developed the Biogeochemical Information Ordering Management Environment (BIOME), a customized search and order system for the World Wide Web (WWW). BIOME is a public system located at http://www-eosdis. ornl.gov/BIOME/biome.html.

CLIMATIC CHANGE↗

Collaborative Data Publication Utilizing the Open Data Repository's Data Publisher

For small communities in multidisciplinary fields such as astrobiology, publishing and sharing data can be challenging. While large, homogenous fields often have repositories and existing data standards, small groups of independent researchers have few options for publishing data that can be utilized within their community. In conjunction with teams at NASA Ames and the University of Arizona, a number of pilot studies are being conducted to assess the needs of these research groups and to guide the software development so that it allows them to publish and share their data collaboratively.

Human-readable interfaces↗

Active Time Domain Reflectometry for Tamper Indication in Unattended Safeguards Systems

The International Atomic Energy Agency (IAEA) continues to expand its use of unattended measurement systems. An increasing number of systems and an expanding family of instruments create challenges in terms of deployment efficiency and the implementation of data authentication measures. In collaboration with the IAEA, tamper-indicating measures to address data-transmission authentication challenges with unattended safeguards systems are under investigation. Pacific Northwest National Laboratory is studying the viability of active time-domain reflectometry (TDR) along two parallel but interconnected paths: (1) swept-frequency TDR as the highly flexible, laboratory gold standard to which field-deployable options can be compared, and (2) a low-cost commercially available spread-spectrum TDR technology as one option for field implementation. This paper describes the TDR methods under investigation and the associated benchtop test-bed, tampering scenarios of interest,, and viability measurement results to date (e.g., comparison of relative sensitivity to tamper scenarios).

Sheen, David M.↗

Complexities in Subsetting Level 2 Data

Satellite Level 2 data presents unique challenges for tools and services. From nonlinear spatial geometry to inhomogeneous file data structure to inconsistent temporal variables to complex data variable dimensionality to multiple file formats, there are many difficulties in creating general tools for Level 2 data support. At NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), we are implementing a general Level 2 Subsetting service for Level 2 data to a user-specified spatio-temporal region of interest (ROI). In this presentation, we will unravel some of the challenges faced in creating this service and the strategies we used to surmount them.

Data acces↗

High-Fidelity, Large-Scale, Realistic Dataset Development

The final report summarizes the work performed for supporting the ARPA-E Grid Optimization Competition (Challenge 2 and Challenge 3) within the stated period. Challenge 2 For the challenge period, the main responsibility of the team is to investigate, gen- erate, and deliver parts of the data sets for the competition, based on the competition model for Challenge 2, existing data sets from Challenge 1, and data source supplied by other data set teams. Challenge 3 For the challenge period, the main responsibility of the team is to propose, create, deliver, and maintain the data format during the competition period. The data format will specify how the benchmark data will be represented and communicated to competitors. It will also specify how competitors should report back the solutions. The data format will be closely aligned with the problem formulation (maintained by the formulation team) and the solution validation process (maintained by the validation team). Our team is also responsible in investigating, generating, and delivering parts of the data sets for the competition. The data sets will be created based on the competition model for Challenge 3, existing data sets from Challenge 1 and Challenge 2, and data source supplied by other data set teams.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Challenges in Evaluating Relationships Between Quantitative Data (Carbon Dioxide) and Qualitative Data (Self-Reported Visual Changes)

Understanding the nuances in clinical data is critical in developing a successful data analysis plan. Carbon dioxide (CO2) data are collected on board the International Space Station (ISS) in a continuous stream. Clinical data on ISS are primarily collected via conversations between individual crewmembers and NASA Flight Surgeons during weekly Private Medical Conferences (PMC). Law, et.al, 20141 demonstrated a statistically significant association between weekly average CO2 levels on ISS and self-reported headaches over the reporting period from March 14, 2001 to May 31, 2012. The purpose of this analysis is to describe the evaluation of a possible association between visual changes and CO2 levels on ISS and to discuss challenges in developing an appropriate analysis plan. METHODS & PRELIMINARY RESULTS: A first analysis was conducted following the same study design as the published work on CO2 and self-reported headaches1; substituting self-reported changes in visual acuity in place of self-reported headaches. The analysis demonstrated no statistically significant association between visual impairment characterized by vision symptoms self-reported during PMCs and ISS average CO2 levels over ISS missions. Closer review of the PMC records showed that vision outcomes are not well-documented in terms of clinical severity, timing of onset, or timing of resolution, perhaps due to the incipient nature of vision changes. Vision has been monitored in ISS crewmembers, pre- and post-flight, using standard optometry evaluations. In-flight visual assessments were limited early in the ISS program, primarily consisting of self-perceived changes reported by crewmembers. Recently, on-orbit capabilities have greatly improved. Vision data ranges from self-reported post-flight changes in visual acuity, pre- to postflight changes identified during fundoscopic examination, and in-flight progression measured by advanced on-orbit clinical imaging capabilities at predetermined testing intervals. In contrast, CO2 data are recorded in a continuous stream over time; however, for the initial analysis this data was categorized into weekly averages.

Mendez, C. M.↗