SEARCH · Search NASA
Results for “Data Mining”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
A Visual Analytics Approach for Correlation, Classification, and Regression Analysis
New approaches that combine the strengths of humans and machines are necessary to equip analysts with the proper tools for exploring today s increasing complex, multivariate data sets. In this paper, a visual data mining framework, called the Multidimensional Data eXplorer (MDX), is described that addresses the challenges of today s data by combining automated statistical analytics with a highly interactive parallel coordinates based canvas. In addition to several intuitive interaction capabilities, this framework offers a rich set of graphical statistical indicators, interactive regression analysis, visual correlation mining, automated axis arrangements and filtering, and data classification techniques. This chapter provides a detailed description of the system as well as a discussion of key design aspects and critical feedback from domain experts.
Profiling Real-Time Electricity Consumption Data for Process Monitoring and Control
Today, smart meters serve as key assets to utilities and their customers because they are capable of recording and communicating real-time energy usage data; thus, enabling better understanding of energy usage patterns. Other potential benefits of smart meters data include the ability to improve customer experience, grid reliability, outage management, and operational efficiency. Despite these tangible benefits, many utilities are inundated by data and remain uncertain about how to extract additional value from these deployed assets outside of billing operations. One way to overcome this challenge is the development of new metrics for classifying utility customers. Traditionally, utilities classified their customers based on their business nature (residential, commercial, and industrial) and/or their total annual consumption. While this classification is useful for some operational functions, it is too limited for designing effective monitoring and control strategies. In this paper, a data mining methodology is proposed for clustering and profiling smart meters data in order to form unique classes of customers exhibiting similar usage patterns. The developed clusters could help utilities in identifying opportunities for achieving some of the benefits of smart meters data.
Automatic Search of Cataclysmic Variables Based on LightGBM in LAMOST-DR7
The search for special and rare celestial objects has always played an important role in astronomy. Cataclysmic Variables (CVs) are special and rare binary systems with accretion disks. Most CVs are in the quiescent period, and their spectra have the emission lines of Balmer series, HeI, and HeII. A few CVs in the outburst period have the absorption lines of Balmer series. Owing to the scarcity of numbers, expanding the spectral data of CVs is of positive significance for studying the formation of accretion disks and the evolution of binary star system models. At present, the research for astronomical spectra has entered the era of Big Data. The Large Sky Area Multi-Object Fiber Spectroscopy Telescope (LAMOST) has produced more than tens of millions of spectral data. the latest released LAMOST-DR7 includes 10.6 million low-resolution spectral data in 4926 sky regions, providing ideal data support for searching CV candidates. To process and analyze the massive amounts of spectral data, this study employed the Light Gradient Boosting Machine (LightGBM) algorithm, which is based on the ensemble tree model to automatically conduct the search in LAMOST-DR7. Finally, 225 CV candidates were found and four new CV candidates were verified by SIMBAD and published catalogs. This study also built the Gradient Boosting Decision Tree (GBDT), Adaptive Boosting (AdaBoost), and eXtreme Gradient Boosting (XGBoost) models and used Accuracy, Precision, Recall, the F1-score, and the ROC curve to compare the four models comprehensively. Experimental results showed that LightGBM is more efficient. The search for CVs based on LightGBM not only enriches the existing CV spectral library, but also provides a reference for the data mining of other rare celestial objects in massive spectral data.
Data Fusion and Mining Techniques to Map Water Use and Drought across Spatial and Temporal Scales
As the world’s water resources come under increasing tension due to dual stressors of climate change and population growth, accurate knowledge of water consumption through evapotranspiration (ET) over a range in spatial scales will be critical in developing adaptation strategies. Remote sensing methods for monitoring consumptive water use (e.g, ET) are becoming increasingly important, especially in areas of significant water and food insecurity. One method to estimate ET from satellite-based methods, the Atmosphere Land Exchange Inverse (ALEXI) model uses the change in mid-morning land surface temperature to estimate the partitioning of sensible and latent heat fluxes which are then used to estimate daily ET. This presentation will outline several recent enhancements to the ALEXI modeling system, with a focus on global ET and drought monitoring. Until recently, ALEXI has been limited to areas with high resolution temporal sampling of geostationary sensors. The use of geostationary sensors makes global mapping a complicated process, especially for real-time applications, as data from as many as five different sensors are required to be ingested and harmonized to create a global mosaic. However, our research team has developed a new and novel method of using twice-daily observations from polar-orbiting sensors such as MODIS and VIIRS to estimate the mid-morning rise in LST that is used to drive the energy balance estimations within ALEXI. This allows the method to be applied globally using a single sensor (in this case, initially MODIS with a planned transition to VIIRS) rather than a global compositing of all available geostationary data. Other advantages of this new method include the higher spatial resolution provided by MODIS and VIIRS and the increased sampling at high latitudes where oblique view angles limit the utility of geostationary sensors. This presentation will focus on global applications for mapping water use and drought using data mining and data fusion across spatial scales extending from 5-km to 30-m “field-scale” estimates.
Polynomial chaos expansions on principal geodesic Grassmannian submanifolds for surrogate modeling and uncertainty quantification
In this work we introduce a manifold learning-based surrogate modeling framework for uncertainty quantification in high-dimensional stochastic systems. Our first goal is to perform data mining on the available simulation data to identify a set of low-dimensional (latent) descriptors that efficiently parameterize the response of the high-dimensional computational model. To this end, we employ Principal Geodesic Analysis on the Grassmann manifold of the response to identify a set of disjoint principal geodesic submanifolds, of possibly different dimension, that captures the variation in the data. Since operations on the Grassmann require the data to be concentrated, we propose an adaptive algorithm based on Riemannian K-means and the minimization of the sample Fréchet variance on the Grassmann manifold to identify “local” principal geodesic submanifolds that represent different system behavior across the parameter space. Polynomial chaos expansion is then used to construct a mapping between the random input parameters and the projection of the response on these local principal geodesic submanifolds. Here, the method is demonstrated on four test cases, a toy-example that involves points on a hypersphere, a Lotka-Volterra dynamical system, a continuous-flow stirred-tank chemical reactor system, and a two-dimensional Rayleigh-Bénard convection problem.
A generalized method for high throughput in-situ experiment data analysis: An example of battery materials exploration
A generalized method is introduced to extract critical information from series of ranked correlated data. The method is generally applicable to all types of spectra evolving as a function of any arbitrary parameter. This approach is based on correlation functions and statistical scedasticity formalism. Numerous challenges in analyzing high throughput experimental data can be tackled using the herein proposed method. We applied this method to understand the reactivity pathway and formation mechanism of a Li-ion battery cathode material during high temperature synthesis using in-situ highenergy X-ray diffraction. We demonstrate that Pearson's correlation function can easily unravel all major phase transition and, more importantly, the minor structural changes which cannot be revealed by conventionally inspecting the series of diffraction patterns. Furthermore, a two-dimensional (2D) reactivity pattern calculated as the scedasticity along all measured reciprocal space of all successive diffraction pattern pairs unveils clearly the structural evolution path and the active areas of interest during the synthesis. The methods described here can be readily used for on-the-fly data analysis during various in-situ operando experiments in order to quickly evaluate and optimize experimental conditions, as well as for post data analysis and large data mining where considerable amount of data hinders the feasibility of the investigation through point-by-point inspection.
Nuclear data uncertainty propagation and modeling uncertainty impact evaluation in neutronics core simulation
Uncertainty analysis is a critical requirement in reactor simulation as it is used to quantify the reliability of best-estimate calculation. A comprehensive uncertainty analysis should characterize all sources of uncertainties in a computationally-feasible and scientifically-defendable manner. Here we employ a well-established reduced order modeling (ROM) based uncertainty quantification methodology to propagate uncertainties throughout neutronic calculations. ROM relies on recent advances in randomized data mining techniques applied to large data streams. In our proposed implementation, the nuclear data uncertainties are first propagated from multi-group level through lattice physics calculation to generate few-group parameter uncertainties, described using a vector of mean values and a covariance matrix. Employing an ROM-based compression of the covariance matrix, the few-group uncertainties are then propagated through downstream core simulation in a computationally efficient manner. This straightforward approach, albeit efficient as compared to brute force forward and/or adjoint-based methods, often employs a number of assumptions that have been unquestioned in the literature of neutronic uncertainty analysis. This manuscript argues that these assumptions could introduce another source of uncertainty referred to as modeling uncertainties, whose magnitude needs to be quantified in tandem with nuclear data uncertainties. Thus, our primary goal is to explore the interactions between these two uncertainty sources in order to assess whether modeling uncertainties have an impact on parameter uncertainties. To explore this endeavor, the impact of a number of modeling assumptions on core attributes uncertainties is quantified. The study employs a CANDU reactor model, with Serpent and NEWT as lattice physics solvers and NESTLE-C as core simulator. The modeling assumptions investigated include those related with the uncertainty propagation method employed, e.g., deterministic vs. stochastic, the few-group energy structure employed to represent the cross-sections, the resonance treatment in lattice physics calculation, the reference values for the cross-section, and the number of samples employed to render ROM compression. Results indicate that some of the modeling assumptions could have a non-negligible impact on the core responses propagated uncertainties, highlighting the need for a more comprehensive approach to combine parameter and modeling uncertainties.
An open experimental database for exploring inorganic materials
The use of advanced machine learning algorithms in experimental materials science is limited by the lack of sufficiently large and diverse datasets amenable to data mining. If publicly open, such data resources would also enable materials research by scientists without access to expensive experimental equipment. Here, we report on our progress towards a publicly open High Throughput Experimental Materials (HTEM) Database (htem.nrel.gov). This database currently contains 140,000 sample entries, characterized by structural (100,000), synthetic (80,000), chemical (70,000), and optoelectronic (50,000) properties of inorganic thin film materials, grouped in >4,000 sample entries across >100 materials systems; more than a half of these data are publicly available. This article shows how the HTEM database may enable scientists to explore materials by browsing web-based user interface and an application programming interface. This paper also describes a HTE approach to generating materials data, and discusses the laboratory information management system (LIMS), that underpin HTEM database. Finally, this manuscript illustrates how advanced machine learning algorithms can be adopted to materials science problems using this open data resource.
PATHA: Performance Analysis Tool for HPC Applications
Large science projects rely on complex workflows to analyze terabytes or petabytes of data. These jobs are often running over thousands of CPU cores and simultaneously performing data accesses, data movements, and computation. It is difficult to identify bottlenecks or to debug the performance issues in these large workflows. In order to address these challenges, we have developed Performance Analysis Tool for HPC Applications (PATHA) using the state-of-art open source big data processing tools. Our framework can ingest system logs to extract key performance measures, and apply the most sophisticated statistical tools and data mining methods on the performance data. Furthermore, it utilizes an efficient data processing engine to allow users to interactively analyze a large amount of different types of logs and measurements. To illustrate the functionality of PATHA, we conduct a case study on the workflows from an astronomy project known as the Palomar Transient Factory (PTF). This study processed 1.6 TB of system logs collected on the NERSC supercomputer Edison. Using PATHA, we were able to identify performance bottlenecks, which reside in three tasks of PTF workflow with the dependency on the density of celestial objects.
Intriguing Trends in Nuclear Physics Articles Authorship
A look at how authorship of physics publications (particularly nuclear publications) have changed throughout the decades by comparing data mined from the National Nuclear Data Center (NNDC) with observations.
Occupant Protection at NASA
This slide presentation reviews NASA's efforts to arrive at protection of occupants of the ORION space craft on landing. An Abbreviated Injury Scale (AIS) has been developed, it is an anatomically-based, consensus-derived, global severity scoring system that classifies each injury by body region according to its relative importance on a 6-point ordinal scale. It reviews an Operationmally Relevant Injury Scale (ORIS), a classification methodology, and shows charts that detail the results of applying this ORIS to the injury databases. One chart uses NASCAR injury classification. It discusses providing a context for the level of risk inherent in the Orion landings in terms that people understand and have a sense for. For example is the risk of injury during an Orion landing roughly the same, better or worse than: An aircraft carrier landing, a NASCAR crash, or a helicopter crash, etc? The data for NASCAR and Indy Racing league (IRL) racing crash and injury data was reviewed. The risk from the Air Force, Navy, and Army injury data was also reviewed. Past NASA and the Soyuz programs injury risks are also reviewed. The work is an attempt to formulate a recommendation to the Orion Project for an acceptable level of injury risk associated with Nominal and Off-Nominal landing cases. The presentation also discusses the data mining and use of the data to Validate NASA Operationally-Relevant Injury Scale (NORIS) / Military Operationally-Relevant Injury Scale (MORIS), developing injury risk criteria, the types of data that are required, NASCAR modeling techniques and crash data, and comparison with the Brinkley model. The development of injury risk curves for each biodynamic response parameter is discussed. One of the main outcomes of this work is to establish an accurate Automated Test Dummy (ATD) that can be used to measure human tolerances.
Technology Alignment and Portfolio Prioritization (TAPP): Advanced Methods in Strategic Analysis, Technology Forecasting and Long Term Planning for Human Exploration and Operations, Advanced Exploration Systems and Advanced Concepts
The Advanced Concepts Office (ACO) at NASA, Marshall Space Flight Center is expanding its current technology assessment methodologies. ACO is developing a framework called TAPP that uses a variety of methods, such as association mining and rule learning from data mining, structure development using a Technological Innovation System (TIS), and social network modeling to measure structural relationships. The role of ACO is to 1) produce a broad spectrum of ideas and alternatives for a variety of NASA's missions, 2) determine mission architecture feasibility and appropriateness to NASA's strategic plans, and 3) define a project in enough detail to establish an initial baseline capable of meeting mission objectives ACO's role supports the decision-making process associated with the maturation of concepts for traveling through, living in, and understanding space. ACO performs concept studies and technology assessments to determine the degree of alignment between mission objectives and new technologies. The first step in technology assessment is to identify the current technology maturity in terms of a technology readiness level (TRL). The second step is to determine the difficulty associated with advancing a technology from one state to the next state. NASA has used TRLs since 1970 and ACO formalized them in 1995. The DoD, ESA, Oil & Gas, and DoE have adopted TRLs as a means to assess technology maturity. However, "with the emergence of more complex systems and system of systems, it has been increasingly recognized that TRL assessments have limitations, especially when considering [the] integration of complex systems." When performing the second step in a technology assessment, NASA requires that an Advancement Degree of Difficulty (AD2) method be utilized. NASA has used and developed or used a variety of methods to perform this step: Expert Opinion or Delphi Approach, Value Engineering or Value Stream, Analytical Hierarchy Process (AHP), Technique for the Order of Prioritization by Similarity to Ideal Solution (TOPSIS), and other multi‐criteria decision-making methods. These methods can be labor-intensive, often contain cognitive or parochial bias, and do not consider the competing prioritization between mission architectures. Strategic Decision-Making (SDM) processes cannot be properly understood unless the context of the technology is understood. This makes assessing technological change particularly challenging due to the relationships "between incumbent technology and the incumbent (innovation) system in relation to the emerging technology and the emerging innovation system." The central idea in technology dynamics is to consider all activities that contribute to the development, diffusion, and use of innovations as system functions. Bergek defines system functions within a TIS to address what is actually happening and has a direct influence on the ultimate performance of the system and technology development. ACO uses similar metrics and is expanding these metrics to account for the structure and context of the technology. At NASA technology and strategy is strongly interrelated. NASA's Strategic Space Technology Investment Plan (SSTIP) prioritizes those technologies essential to the pursuit of NASA's missions and national interests. The SSTIP is strongly coupled with NASA's Technology Roadmaps to provide investment guidance during the next four years, within a twenty-year horizon. This paper discusses the methods ACO is currently developing to better perform technology assessments while taking into consideration Strategic Alignment, Technology Forecasting, and Long Term Planning.
Validation of A Simulation Environment for Future Space Traffic Management
This study presents initial results of a newly developed simulation environment intendedto explore and assess future Space Traffic Management (STM) scenarios. The number ofnew Resident Space Objects (RSOs) in near-Earth orbit is expected to increase significantlywith the expected future deployment of a number of large constellations. These futurescenarios involve the addition of many tens of thousands of new RSOs, making the analysisinto their impact on collision risk extend beyond what traditional data-mining of present-dayconjunction data can reliably predict. To address this, a robust simulation environment wasdeveloped that implements a full force-model for orbit propagation, and computes continuousall-on-all conjunction statistics for arbitrarily large catalog sizes and simulation timeframes.Collision avoidance and station-keeping maneuvers can be optionally implemented based onconfigurable user inputs including physical characteristics and spacecraft meta-data (e.g.,commercial/government, owner country, etc.). Constellation build-out and de-orbit scenarioswere also implemented and modeled based on real-data analysis. Validation of the simulationresults was a critical component of the simulation development, and was done using the currentcatalog with comparisons against both public and internal NASA data-sets. The comparisonsdemonstrate that, with the appropriate settings, representative levels of conjunction ratesand probabilities can be obtained, providing confidence that the simulation tool can generatemeaningful outcomes for test scenarios. As an initial demonstration of the tool’s capabilities,year-long simulations with station-keeping were conducted to examine conjunction historiesusing both the current object catalog (5800 active satellites) and a hypothetical 60,000 objectscenario involving five potential large constellations. Output metrics include the number ofconjunction events, estimates of collision consequence (fragmentation), delta-V maneuver costs,and the probability of at least one collision occurring. The results highlight the potential thatthe simulation tool has for incorporating and running performance comparisons between, e.g.,various sets of maneuver guidelines, industry norms, and definitions of risk, with the overallobjective of providing actionable data to STM policy makers. The presentation will providean overview of the simulation development and validation efforts, as well as a discussion ofobservations gathered from the initial simulations performed.
Clustering Days with Similar Airport Weather Conditions
On any given day, traffic flow managers must often rely on past experience and intuition when developing traffic flow management initiatives that mitigate imbalances between the aircraft demand and the weather impacted airport capacity. The goal of this study was to build on recent efforts to apply data mining classification and clustering algorithms to vast archives of historical weather and air traffic data to identify patterns and past decisions that can ultimately inform day-of-operations decision-making. More specifically, this study identified similar weather impacted days at select U.S. airports, and analyzed the traffic management initiatives implemented on these representative days. The identification of the similar days was accomplished by applying a decision tree algorithm to the hourly Localized Aviation Model Output Statistics Program observations and the arrival delays for Newark Liberty International Airport. The branches from the trained decision tree were subsequently pruned to identify four weather conditions that resulted in medium to high delays for the arrivals scheduled to Newark in 2012. Using these weather conditions, four, daily airport-level Weather Impacted Traffic Index values were calculated using the Localized Aviation Model Output Statistics Program observations and the 2012 scheduled arrival counts from the FAAs Aviation System Performance Metric system. The four, daily Weather Impacted Traffic Index values for 2012 were subsequently clustered using an Expectation Maximization clustering algorithm, and nine unique types of weather days at Newark were identified. By far the most prominent type of day at Newark was a day associated with relatively good weather conditions, where there was little convective activity, winds were low, ceilings and visibility were high and there was little precipitation. Moderate levels of convective activity characterized the next most prominent type of day. Days with persistently high winds or low ceiling and visibility levels were relatively rare in 2012. Lastly, the frequency at which Ground Delay Programs, Ground Stops and Miles-in-Trail restrictions were implemented on each of the typical types of days at Newark were analyzed. Based on the results, it does appear as if the usage of Miles-in-Trail, Ground Delay Program and Ground Stop restrictions correlates well with the severity of the weather associated with each unique type of weather impacted day at Newark. Furthermore, the results demonstrate that it is feasible to use historical weather and air traffic archives to provide guidance on the types of traffic management restrictions to implement in response to the weather conditions impacting an airport.
Clustering Days with Similar Airport Weather Conditions
On any given day, traffic flow managers must often rely on past experience and intuition when developing traffic flow management initiatives that mitigate imbalances between the aircraft demand and the weather impacted airport capacity. The goal of this study was to build on recent efforts to apply data mining classification and clustering algorithms to vast archives of historical weather and air traffic data to identify patterns and past decisions that can ultimately inform day-of-operations decision-making. More specifically, this study identified similar weather impacted days at select U.S. airports, and analyzed the traffic management initiatives implemented on these representative days. The identification of the similar days was accomplished by applying a decision tree algorithm to the hourly Localized Aviation Model Output Statistics Program observations and the arrival delays for Newark Liberty International Airport. The branches from the trained decision tree were subsequently pruned to identify four weather conditions that resulted in medium to high delays for the arrivals scheduled to Newark in 2012. Using these weather conditions, four, daily airport-level Weather Impacted Traffic Index values were calculated using the Localized Aviation Model Output Statistics Program observations and the 2012 scheduled arrival counts from the FAAs Aviation System Performance Metric system. The four, daily Weather Impacted Traffic Index values for 2012 were subsequently clustered using an Expectation Maximization clustering algorithm, and nine unique types of weather days at Newark were identified. By far the most prominent type of day at Newark was a day associated with relatively good weather conditions, where there was little convective activity, winds were low, ceilings and visibility were high and there was little precipitation. Moderate levels of convective activity characterized the next most prominent type of day. Days with persistently high winds or low ceiling and visibility levels were relatively rare in 2012. Lastly, the frequency at which Ground Delay Programs, Ground Stops and Miles-in-Trail restrictions were implemented on each of the typical types of days at Newark were analyzed. Based on the results, it does appear as if the usage of Miles-in-Trail, Ground Delay Program and Ground Stop restrictions correlates well with the severity of the weather associated with each unique type of weather impacted day at Newark. Furthermore, the results demonstrate that it is feasible to use historical weather and air traffic archives to provide guidance on the types of traffic management restrictions to implement in response to the weather conditions impacting an airport.
RNAV STAR Procedural Adherence
Flight crews and air traffic controllers have reported many safety concerns regarding area navigation standard terminal arrival routes (RNAV STARs). However, our information sources to quantify these issues are limited to subjective reporting and time consuming case-by-case investigations. This work is a preliminary study into the objective performance of instrument procedures and provides a framework to track procedural concepts and assess design functionality. We created a tool and analysis methods for gauging aircraft adherence as it relates to RNAV STARs. This information is vital for comprehensive understanding of how our air traffic behaves. In this exploratory archival study, we mined the performance of 24 major US airports over the preceding three years. Overlaying radar track data on top of RNAV STAR routes provided a comparison between aircraft flight paths and the waypoint positions and altitude restrictions. NASA Ames Supercomputing resources were utilized to perform the data mining and processing. We assessed STARs by lateral transition path (full-lateral), vertical restrictions (full-lateralfull-vertical), and skipped waypoints (skips). In addition, we graphed aircraft altitudes relative to the altitude restrictions and their occurrence rates. Full-lateral adherence was generally greater than Full-lateralfull-vertical, but the difference between the rates was not always consistent. Full-lateralfull-vertical adherence medians of the 2016 procedures ranged from 0 in KDEN (Denver) to 21 in KMEM (Memphis). Waypoint skips ranged from 0 to nearly 100 for specific waypoints. Altitudes restrictions were sometimes missed by systematic amounts in 1000 ft. increments from the restriction, creating multi-modal distributions. Other times, altitude misses looked to be more normally distributed around the restriction. This tool may aid in providing acceptability metrics as well as risk assessment information.