SEARCH · Search NASA
Results for “correlation mining”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
The Role of Data Filtering in Open Source Software Ranking and Selection
Faced with more than 100M open source projects, a more manageable small subset is needed for most empirical investigations. More than half of the research papers in leading venues investigated filtering projects by some measure of popularity with explicit or implicit arguments that unpopular projects are not of interest, may not even represent "real" software projects, or that less popular projects are not worthy of study. However, such filtering may have enormous effects on the results of the studies if and precisely because the sought-out response or prediction is in any way related to the filtering criteria.This paper exemplifies the impact of this common practice on research outcomes, specifically how filtering of software projects on GitHub based on inherent characteristics affects the assessment of their popularity. Using a dataset of over 100,000 repositories, we used multiple regression to model the number of stars -a commonly used proxy for popularity- based on factors such as the number of commits, the duration of the project, the number of authors and the number of core developers. Our control model included the entire dataset, while a second filtered model considered only projects with ten or more authors. The results indicated that while certain characteristics of the repository consistently predict popularity, the filtering process significantly alters the relationships between these characteristics and the response. We found that the number of commits exhibited a positive correlation with popularity in the control sample but showed a negative correlation in the filtered sample. These findings highlight the potential biases introduced by data filtering and emphasize the need for careful sample selection in empirical research of mining software repositories. We recommend that empirical work should either analyze complete datasets such as World of Code, or employ stratified random sampling from a complete dataset to ensure that filtering is not biasing the results.
Understanding the Hierarchy of Dense Subgraphs in Stationary and Temporally Varying Setting
Graphs are widely used to model relationships in a wide variety of domains such as sociology, bioinformatics, infrastructure, the WWW, to name a few. One of the key observations is that while real-world graphs are often globally sparse, they are locally dense. In other words, the average degree is often quite small (say at most 10 in a million vertex graph), but vertex neighborhoods are often dense. Finding dense subgraphs is a critical aspect of graph mining It has been used for finding communities and spam link farms in web graphs, graph visualization, real-time story identification, DNA motif detection in biological networks, finding correlated genes, epilepsy prediction, finding price value motifs in financial data, graph compression, distance query indexing, and increasing the throughput of social networking site servers. However, most standard formulations of this problem (like clique, quasi-clique, k-densest subgraph) are NP-hard. Furthermore, current dense subgraph finding algorithms usually optimize some objective, and only find a few such subgraphs without providing any structural relations, whereas the goal is rarely to find the "true optimum," but to identify many (if not all) dense substructures, understand their distribution in the graph, and ideally determine relationships among them. In this project, we first aim to devise algorithms and provide 3 implementations with nice visualizations to find the hierarchy between dense subgraphs, and then understand the structure of the hierarchy to gain more insight on the hidden patterns in real-world networks. Another important aspects in graph analysis is the temporal nature of networks. Networks evolve over time and in many applications data arrives at a high velocity, and thus it is important to design algorithms that can process data efficiently. We report three main results towards identifying dense structures in large evolving graphs. First, we will show how the hierarchical connectedness structure can be maintained efficiently, where connectedness is defined by increasing levels of connectivity strength. Next, we present dense structure can be identified in bipartite graphs without building projection graphs. And finally, we present a new method for peeling algorithms This new approach avoids sequential nature of peeling algorithms and is amenable to parallelization, which is crucial for processing high velocity data.
Data Mining for Parameters Affecting Polymorph Selection in Contorted Hexabenzocoronene Derivatives
The macroscopic properties of molecular materials can be drastically influenced by their solid-state packing arrangements, of which there can be many (e.g., polymorphism). Strategies to controllably and predictively access select polymorphs are thus highly desired, but computationally predicting the conditions necessary to access a given polymorph is challenging with the current state of the art. Using derivatives of contorted hexabenzocoronene, cHBC, we employed data mining, rather than first-principles approaches, to find relationships between the crystallizing molecule, postdeposition solvent-vapor annealing conditions that induce polymorphic transformation, and the resulting polymorphs. This analysis yields a correlative function that can be used to successfully predict the appearance of either one of two polymorphs in thin films of cHBC derivatives. Within the postdeposition processing phase space of cHBC derivatives, we have demonstrated an approach to generate guidelines to select crystallization conditions to bias polymorph access. Finally, we believe this approach can be applied more broadly to accelerate the predictions of processing conditions to access desired molecular polymorphs, making progress toward one of the grand challenges identified by the Materials Genome Initiative.
Modern sensitivity analysis - 15651
Sensitivity analysis has been applied to complex models, such as performance assessment models, since at least the advent of computational tools to support such an approach to model evaluation. Initially, sensitivity analysis methods were applied to deterministic models and focused on changing one input variable at a time. Probabilistic modeling opened the door to evaluating changes in all input variables simultaneously. Standardized rank regression methods were primarily used in the last century to evaluate probabilistic models. These methods assume linearity in the rank response of the model to the inputs. At the very least this assumes monotonicity in the response, but for the most part the response needs to be near-linear. This is highly unlikely for performance assessment models that are based on differential equations, multiplication and division of variables that are usually not correlated, and conservatism that is not well understood. There are better statistical methods for evaluating highly non-linear and dynamic probabilistic models. Some of these methods were founded in the data mining industry. This century has seen a rapid development of the internet that led directly to the development of data mining techniques for massive databases. Although the goals were different, these techniques can be adapted to sensitivity analysis for complex probabilistic models, which, also, form large, or even massive, databases. These modern sensitivity analysis procedures assess the importance of every explanatory variable (input parameters) used in a probabilistic model. In practice, this means that for each endpoint of interest, every explanatory variable in the model is associated with a quantitative measure of importance. For a given explanatory variable, the importance value depends on the endpoint of interest. This global approach to sensitivity analysis allows all levels of interactions to be evaluated, so that the variation in the output can be measured at every level of possible interaction. The effects are collected together during sensitivity analysis processing, and then separated out to attribute the overall contribution of each input parameter to the model output. These global sensitivity analysis procedures have the ability to characterize non-linear and non-monotonic relationships between explanatory variables and endpoints of interest. This is critically important because of the need to characterize complex interactions among multiple explanatory variables in PA models that often have non-linear and non-monotonic effects. The options for sensitivity analysis are fully explored, including an evaluation of several methods when applying them to a well-known complex non-linear non-monotonic model. The results clearly demonstrate the relative benefit of using data mining methods over simple statistical approaches such as standardized rank regression. (authors)
Deep data mining in a real space: Separation of intertwined electronic responses in a lightly doped BaFe 2 As 2
Electronic interactions present in material compositions close to the superconducting dome play a key role in the manifestation of high-T c superconductivity. In many correlated electron systems, however, the parent or underdoped states exhibit strongly inhomogeneous electronic landscape at the nanoscale that may be associated with competing, coexisting, or intertwined chemical disorder, strain, magnetic, and structural order parameters. Here we demonstrate an approach based on a combination of scanning tunneling microscopy/spectroscopy and advanced statistical learning for an automatic separation and extraction of statistically significant electronic behaviors in the spin density wave regime of a lightly (~1%) gold-doped BaFe 2 As 2 . Lastly, we show that the decomposed STS spectral features have a direct relevance to fundamental physical properties of the system, such as SDW-induced gap, pseudogap-like state, and impurity resonance states.
Mining data in a dynamic PRA framework
Computational (also known as Dynamic) Probabilistic Risk Assessment (PRA) methods employs system simulation codes coupled with stochastic analysis tools in order to determine probabilities of certain outcomes such as system failure. In contrast to classical PRA methods (i.e., Event-Tree and Fault-Tree) in which timing and sequencing of events is set by the analyst, accident progression is dictated by the system control logic and its interaction with system temporal evolution. Due to the nature of the problem, Simulation-Based PRA methods require large amount of computationally expensive simulation runs. In addition, these methods usually generate a large number of simulation runs (database storage may be on the order of gigabytes or higher). In this paper we investigate and apply several methods and algorithms to analyze these large amounts of time-dependent data. Furthermore, the scope of this article is to present a broad overview of methods and algorithms that can be used to improve data quality and to analyze and extract information from large data sets containing time dependent data. In this context, “extracting information” means constructing input-output correlations, finding commonalities, and identifying outliers.
Chemical and isotope compositions of shallow groundwater in areas impacted by hydraulic fracturing and surface mining in the Central Appalachian Basin, Eastern United States
Here, hydraulic fracturing of shale deposits has greatly increased the productivity of the natural gas industry by allowing it to exploit previously inaccessible reservoirs. Previous research has demonstrated that this practice has the potential to contaminate shallow aquifers with methane (CH 4 ) from deeper formations. This study compares concentrations and isotopic compositions of CH 4 sampled from domestic groundwater wells in Letcher County, Eastern Kentucky in order to characterize its occurrence and origins in relation to both neighboring hydraulically fractured natural gas wells and surface coal mines. The studied groundwater showed concentrations of CH 4 ranging from 0.05 mg/L to 10 mg/L, thus, no immediate remediation is required. The δ 13 C values of CH 4 ranged from 66‰ to 16‰, and δ 2 H values ranged from –286‰ to –86‰, suggesting an immature thermogenic and mixed biogenic/thermogenic origin. The occurrence of CH 4 was not correlated with proximity to hydraulically fractured natural gas wells. Generally, CH 4 occurrence corresponded with groundwater abundant in Na + , Cl – , and HCO 3 – , and with low concentrations of SO 4 –2 . The CH 4 and SO 4 –2 concentrations were best predicted by the oxidation/reduction potential of the studied groundwater. CH 4 was abundant in more reducing waters, and SO 4 –2 was abundant in more oxidizing waters. Additionally, groundwater in greater proximity to surface mining was more likely to be oxidized. This, in turn, might have increased the likelihood of CH 4 oxidation in shallow groundwater.
Multi-configuration Membrane Distillation Model (MCMD)
Many membrane distillation models have been created to simulate the heat and mass exchange process involved but most of the literature only validates models to a couple of cases with minor configuration changes. Tools are needed that allow tradeoffs between many configurations. The multiconfiguration membrane distillation model handles many configurations. This report introduces membrane distillation, provides theory, and presents the work to verify and validate the model against experimental data from Colorado School of Mines and a lower resolution model created at the National Renewable Energy Laboratory. Though more data analysis and testing are needed, an initial look at the model to experimental comparisons indicates that the model correlates to the data well but that design comparisons are likely to be incorrect across a broad range of configurations. More accurate quantification of heat and mass transfer through computational fluid mechanics is suggested.
Data Mining of Polymer Phase Transitions upon Temperature Changes by Small and Wide-Angle X-ray Scattering Combined with Raman Spectroscopy
The complex physical transformations of polymers upon external thermodynamic changes are related to the molecular length of the polymer and its associated multifaceted energetic balance. The understanding of subtle transitions or multistep phase transformation requires real-time phenomenological studies using a multi-technique approach that covers several length-scales and chemical states. A combination of X-ray scattering techniques with Raman spectroscopy and Differential Scanning Calorimetry was conducted to correlate the structural changes from the conformational chain to the polymer crystal and mesoscale organization. Current research applications and the experimental combination of Raman spectroscopy with simultaneous SAXS/WAXS measurements coupled to a DSC is discussed. In particular, we show that in order to obtain the maximum benefit from simultaneously obtained high-quality data sets from different techniques, one should look beyond traditional analysis techniques and instead apply multivariate analysis. Data mining strategies can be applied to develop methods to control polymer processing in an industrial context. Crystallization studies of a PVDF blend with a fluoroelastomer, known to feature complex phase transitions, were used to validate the combined approach and further analyzed by MVA.
Pahute Mesa-Oasis Valley Hydrostratigraphic Framework Model for Corrective Action Units 101 and 102: Central and Western Pahute Mesa, Nye County, Nevada (Rev. 0)
A new, revised three-dimensional (3-D) hydrostratigraphic framework model (HFM) for the Central and Western Pahute Mesa Corrective Action Units 101 and 102 was completed in 2019. The initial (Phase I) model was completed in 2002 and was documented in A Hydrostratigraphic Model and Alternatives for the Groundwater Flow and Contaminant Transport Model of Corrective Action Units 101 and 102: Central and Western Pahute Mesa, Nye County, Nevada (BN, 2002). Subsequent flow and transport modeling revealed the need to collect additional data to reduce uncertainties. The Phase II data collection and characterization effort included extensive well drilling and testing activities, and rebuilding of the Phase I 3-D HFM, resulting in the Phase II HFM (NSTec, 2014). Further drilling (Well ER-20-12), data collection, and interpretive work (e.g., Thirsty Canyon Lineament [Wurtz and Day, 2018]) were incorporated into the Pahute Mesa-Oasis Valley (PM-OV) HFM. The PM-OV HFM incorporates the area of interest from the Phase II HFM and adds additional areas, primarily to the north and west with minor additions to the south and east. The new, expanded area matches the PM-OV groundwater basin as defined in Fenelon et al. (2016) and includes Pahute Mesa, a former nuclear testing area at the Nevada National Security Site; and Oasis Valley, a groundwater discharge area downgradient from contaminant source areas on Pahute Mesa. The data and information necessary to build the PM-OV HFM are provided in Navarro (2019). The PM-OV HFM will be used by hydrologic modelers who are tasked with developing a model to determine how contaminants are transported by groundwater flow in an area of complex geology. The model area is large (more than 6,250 square kilometers) and geologically complex, including Paleozoic- to Mesozoic-age structures (e.g., thrust faults, Thirsty Canyon Lineament), at least seven Tertiary-age calderas, several intrusive bodies, and many relatively recent basin-and-range normal faults. Investigators from the U.S. Geological Survey; National Security Technologies, LLC; National Laboratories; and U.S. Department of Energy contractors have organized the more than 300 volcanic units and other rocks in the model area into 77 hydrostratigraphic units (HSUs). The volcanic rocks are subdivided into 42 aquifers, 11 confining units, and 9 composite units (containing both aquifer and confining unit rocks). The underlying pre-Tertiary rocks were divided into 5 HSUs, comprising 2 aquifers and 3 confining units. The model also includes 1 alluvial aquifer unit, 8 intrusive confining units, and 1 granite confining unit. The model depicts the thickness, extent, and geometric relationships of these HSUs (“layers” in the model) along with all the major structural features that control them, including calderas and faults. The most substantive differences between the Phase II HFM and the PM-OVHFM include (1) addition of the new data and resulting revised interpretations from the additional area encompassed by the PM-OV groundwater basin; (2) incorporation of 16 new faults; (3) 12 boreholes from the northern extension; (4) 2 additional caldera-collapse collars; and (5) correlation of the simulated faulting and hydrostratigraphy in the eastern part of the model domain with the delineations represented in the neighboring Rainier Mesa-Shoshone Mountain HFM-Prime and Yucca Flat-Climax Mine HFM. All PM-OV HFM delineations were based on published and unpublished data from geophysical investigations, surface geologic maps, and drill-hole records. Phase II approached model uncertainty using the model-building software to test model configurations and hydrologic consequences using a range of values for parameters such as fault offset amounts and HSU elevations. The PM-OV HFM expands on this approach to encompass the entire PM-OV groundwater basin.
Full-flow-regime storage-streamflow correlation patterns provide insights into hydrologic functioning over the continental US
Interannual changes in low, median, and high regimes of streamflow have important implications for flood control, irrigation, and ecologic and human health. The Gravity Recovery and Climate Experiment (GRACE) satellites record global terrestrial water storage anomalies (TWSA), providing an opportunity to observe, interpret, and potentially utilize the complex relationships between storage and full-flow-regime streamflow. Here we show that utilizable storage-streamflow correlations exist throughout vastly different climates in the continental US (CONUS) across low- to high-flow regimes. A panoramic framework, the storage-streamflow correlation spectrum (SSCS), is proposed to examine macroscopic gradients in these relationships. SSCS helps form, corroborate or reject hypotheses about basin hydrologic behaviors. SSCS patterns vary greatly over CONUS with climate, land surface, and geologic conditions. Data mining analysis suggests that for catchments with hydrologic settings that favor storage over runoff, e.g., a large fraction of precipitation as snow, thick and highly-permeable permeable soil, SSCS values tend to be high. Based on our results, we form the hypotheses that groundwater flow dominates streamflows in South-eastern CONUS and Great Plains, while thin soils in a belt along the Appalachian Plateau impose a limit on water storage. SSCS also suggests shallow water table caused by high-bulk density soil and flat terrain induces rapid runoff in several regions. Our results highlight the importance of subsurface properties and groundwater flow in capturing flood and drought. Finally, we propose that SSCS can be used as a fundamental hydrologic signature to constrain models and to provide insights that lead us to better understand hydrologic functioning.
Review of High‐Speed Digital Image Correlation: Advancements and Good Practices
This paper reviews the current state of the art in high‐speed (HS) and ultrahigh‐speed (UHS) digital image correlation (DIC) techniques, emphasizing their critical role in experimental research across various scientific domains. HS and UHS DIC have evolved significantly, driven by advancements in camera systems, image processing algorithms and experimental methodologies. These developments have opened new avenues for capturing and analysing dynamic events with unprecedented temporal and spatial detail, but not without introducing challenges such as optical distortions, motion blur and lighting issues that can affect measurement quality. This review advocates for standardized reporting practices in HS/UHS DIC methodologies to improve reproducibility and reliability across studies, drawing on guidelines from the International Digital Image Correlation Society (iDICs). Through a comprehensive analysis of over 150 articles, this review identifies key advancements in imaging technology and their application in six research domains: material characterization, test development, fracture mechanics, model validation, ballistic and explosive phenomena assessment and measurement uncertainties. Distinctions between two‐dimensional (2D) and stereo‐DIC applications are explored, offering insights into their practical implementation and trade‐offs. Good practices for HS/UHS DIC applications are proposed along with suggestions for future directions for this evolving field, highlighting the indispensable role of technological innovation in expanding the capabilities of optical metrology.
Developing reservoir monthly inflow forecasts using artificial intelligence and climate phenomenon information
Reservoirs are fundamental human–built infrastructures that collect, store, and deliver fresh surface water in a timely manner for many purposes. Efficient reservoir operation requires policy makers and operators to understand how reservoir inflows are changing under different hydrological and climatic conditions to enable forecast–informed operations. Over the last decade, the uses of Artificial Intelligence and Data Mining [AI & DM] techniques in assisting reservoir streamflow subseasonal to seasonal forecasts have been increasing. In this study, Random Forest [RF), Artificial Neural Network (ANN), and Support Vector Regression (SVR) are employed and compared with respect to their capabilities for predicting 1 month–ahead reservoir inflows for two headwater reservoirs in USA and China. Both current and lagged hydrological information and 17 known climate phenomenon indices, i.e., PDO and ENSO, etc., are selected as predictors for simulating reservoir inflows. Here, the results show three methods are capable of providing monthly reservoir inflows with satisfactory statistics; the results obtained by Random Forest have the best statistical performances compared with the other two methods; another advantage of Random Forest algorithm is its capability of interpreting raw model inputs; climate phenomenon indices are useful in assisting monthly or seasonal forecasts of reservoir inflow; and different climate conditions are autocorrelated with up to several months, and the climatic information and their lags are cross correlated with local hydrological conditions in our case studies.
Comparative Assessment of Data-driven Process Models in Health Information Technology
Process mining for conformance analysis consists of comparing a reference process model against a data-driven process model generated via log files from information technology systems. However, in the absence of a complete reference process model, we found no suggested approaches in the literature to address the need for evaluating process conformance among different healthcare facilities to assess standardization of care. Our goal is to find similarities and dissimilarities in data-driven process models among US Veterans Health Administration (VHA) facilities that can be indicative of patient safety issues. Our hypothesis was that the analysis would not produce statistically significant differences in outcome. We present a unique implementation of conformance analysis in process mining that consists of combining process mining, process mapping and statistical metrics. We illustrate our approach by applying it to the analysis of two clinical radiology order process models generated from healthcare data provided by two similar facilities in the VHA. The comparative assessment showed that about 70% of the orders completed successfully and 30% were not completed due to policy and duplications. Our analysis found a good statistical correlation between both facilities, as the Spearman’s correlation coefficient between facilities for the frequency of cases per total hours was 0.87879, for the frequency of cases by state transition was 0.79702 and for the throughput time per state transition was 0.63582. Additional statistical analyses using the Mann-Whitney U test and the root mean square error both produced values that were not significant. The foregoing approach validated our hypothesis by demonstrating a good statistical correlation of data describing the flow of clinical radiology orders absent a credible reference model. Finding good agreement between both facilities was important in confirming that the clinical orders flow in a similar manner, suggesting standardization of care.
Effective Long-term Monitoring Strategies by Integrating Reactive Transport Models with In situ Geochemical Measurements - 16212
Long-term monitoring of contaminant transport in groundwater is expected to account for a large fraction of future life-cycle cleanup costs at the DOE sites. The Attenuation-Based Remedies in the Subsurface Applied Field Research Initiative (ABRS AFRI) is developing an innovative approach for cost-effective in situ long-term monitoring. The approach is based on strategically adding the measurements of controlling variables (such as pH, redox potential, electrical conductivity, and groundwater level), which control the plume mobility and its spatial and temporal distribution. In situ measurements of these variables - supplemented with a reduced number of standard periodic groundwater samples - are expected to lead to more cost-effective monitoring, and can also serve as an early warning system for detecting unexpected plume migration. The objective of this study is to support the development of such a long-term monitoring approach through the application of advanced computational methods, including (1) statistical data mining and analysis, and (2) three-dimensional flow and reactive chemical transport modeling. We have performed the statistical data analysis of historical and current monitoring data at the Savannah River Site (SRS) F-Area to quantify the correlations between the controlling variables and radioactive contaminant concentrations. In parallel, we have used 3D modeling of flow and contaminant transport to provide the prediction of the contaminant plume evolution. The results of both data analysis and modeling confirmed that the correlations between controlling variables and contaminant concentrations are significant. Modeling results also suggest that the correlation parameters will change in the future, which is important to assess the long-term efficacy of the proposed monitoring approach. (authors)
Update on NREL Outdoor Exposure Campaign of Solar Mirrors
Beginning in 2018, the National Renewable Energy Laboratory restarted exposure campaigns on new and archived samples as part of a multiyear project, some with outdoor exposure for more than a decade. By resuming exposure and collecting and analyzing data on thousands of samples going back decades, several goals can be advanced that can be difficult to determine within the timeline of most projects: 1) correlating an accelerated exposure campaign to outdoor aging, specifically with xenon arc lamp exposure chambers; 2) drawing conclusions between specific corrosion mechanisms and weather patterns; and 3) finding novel relationships between mirror composition and performance. In addition to building and mining a database, we will experiment with new characterization techniques, primarily focused on macroscopic and microscopic imaging. In introducing these techniques more broadly, it may be possible to reveal a more direct line between optical performance and exposure campaigns by better understanding the degradation mechanisms occurring.
Generation of a fluorescent short-lifetime oxygen nanosensor by coupling of an exciplex forming dye pair
Typical oxygen-sensitive dyes have lifetimes in the μs-ms regime for sensitive oxygen detection based on molecular quenching. Due to the long triplet-state lifetimes and poor quenching efficiency of these probes, it has been difficult to develop short lifetime probes. We have developed oxygen sensors using a short-lifetime dye (∼30–60 ns) that forms an anthracene-aniline exciplex, which is quenched by molecular oxygen through an excited-state charge transfer mechanism. To achieve measurements in aqueous systems, we have tethered the dyes together via a C3 alkyl linker to facilitate greater exciplex formation at low dye concentration (20 μg mL −1 ) and encapsulated the resulting compound into polymeric nanoparticles to form an aqueous-dispersible short-lifetime oxygen nanosensor. The sensors emit at 420 nm (unfolded dye) and 500 nm (exciplex), and the ratiometric Stern-Volmer Constant is 1.52 ± 0.02 L mol −1 , showing good sensitivity from 0 % to 40 % dissolved oxygen without the tradeoff of a long lifetime needed for direct molecular quenching. In this paper, we characterize this sensor for both lifetime-based measurements using time-correlated single-photon counting and ratiometric intensity-based measurements using the unfolded dye emission as an internal reference peak.