Search NASA⌕ Search

SEARCH · Search NASA

Results for “data intensive computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,117 records · Page 62
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

Autonomous METLIN-Guided In-source Fragment Annotation for Untargeted Metabolomics

Computational metabolite annotation in untargeted profiling aims at uncovering neutral molecular masses of underlying metabolites and assign those with putative identities. Existing annotation strategies rely on the observation and annotation of adducts to determine metabolite neutral masses. However, a significant fraction of features usually detected in untargeted experiments remains unannotated, which limits our ability to determine neutral molecular masses. Despite the availability of tools to annotate, relatively few of them benefit from the inherent presence of in-source fragments in liquid chromatography-electrospray ionization-mass spectrometry. Here, we introduce a strategy to annotate in-source fragments in untargeted data using low-energy tandem mass spectrometry (MS) spectra from the METLIN library. Our algorithm, MISA (METLIN-guided in-source annotation), compares detected features against low-energy fragments from MS/MS spectra, enabling robust annotation and putative identification of metabolic features based on low-energy spectral matching. The algorithm was evaluated through an annotation analysis of a total of 140 metabolites across three different sets of biological samples analyzed with liquid chromatography-mass spectrometry. Results showed that, in cases where adducts were not formed or detected, MISA was able to uncover neutral molecular masses by in-source fragment matching. MISA was also able to provide putative metabolite identities via two annotation scores. These scores take into account the number of in-source fragments matched and the relative intensity similarity between the experimental data and the reference low-energy MS/MS spectra. Overall, results showed that in-source fragmentation is a highly frequent phenomena that should be considered for comprehensive feature annotation. Thus, combined with adduct annotation, this strategy adds a complementary annotation layer, enabling in-source fragments to be annotated and increasing putative identification confidence. The algorithm is integrated into the XCMS Online platform and is freely available at http://xcmsonline.scripps.edu.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2015

Compressed-sensing-based content-driven hierarchical reconstruction: Theory and application to C-arm cone-beam tomography

Purpose: This paper addresses the reconstruction of x-ray cone-beam computed tomography (CBCT) for interventional C-arm systems. Subsampling of CBCT is a significant issue with C-arms due to their slow rotation and to the low frame rate of their flat panel x-ray detectors. The aim of this work is to propose a novel method able to handle the subsampling artifacts generally observed with analytical reconstruction, through a content-driven hierarchical reconstruction based on compressed sensing. Methods: The central idea is to proceed with a hierarchical method where the most salient features (high intensities or gradients) are reconstructed first to reduce the artifacts these features induce. These artifacts are addressed first because their presence contaminates less salient features. Several hierarchical schemes aiming at streak artifacts reduction are introduced for C-arm CBCT: the empirical orthogonal matching pursuit approach with the ℓ{sub 0} pseudonorm for reconstructing sparse vessels; a convex variant using homotopy with the ℓ{sub 1}-norm constraint of compressed sensing, for reconstructing sparse vessels over a nonsparse background; homotopy with total variation (TV); and a novel empirical extension to nonlinear diffusion (NLD). Such principles are implemented with penalized iterative filtered backprojection algorithms. For soft-tissue imaging, the authors compare the use of TV and NLD filters as sparsity constraints, both optimized with the alternating direction method of multipliers, using a threshold for TV and a nonlinear weighting for NLD. Results: The authors show on simulated data that their approach provides fast convergence to good approximations of the solution of the TV-constrained minimization problem introduced by the compressed sensing theory. Using C-arm CBCT clinical data, the authors show that both TV and NLD can deliver improved image quality by reducing streaks. Conclusions: A flexible compressed-sensing-based algorithmic approach is proposed that is able to accommodate for a wide range of constraints. It is successfully applied to C-arm CBCT images that may not be so well approximated by piecewise constant functions.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2016

MO-DE-207B-05: Predicting Gene Mutations in Renal Cell Carcinoma Based On CT Imaging Features: Validation Using TCGA-TCIA Datasets

Purpose: The goal of this work is to investigate the use of contrast enhanced computed tomographic (CT) features for the prediction of mutations of BAP1, PBRM1, and VHL genes in renal cell carcinoma (RCC). Methods: For this study, we used two patient databases with renal cell carcinoma (RCC). The first one consisted of 33 patients from our institution (UT Southwestern Medical Center, UTSW). The second one consisted of 24 patients from the Cancer Imaging Archive (TCIA), where each patient is connected by a unique identi?er to the tissue samples from the Cancer Genome Atlas (TCGA). From the contrast enhanced CT image of each patient, tumor contour was first delineated by a physician. Geometry, intensity, and texture features were extracted from the delineated tumor. Based on UTSW dataset, we completed feature selection and trained a support vector machine (SVM) classifier to predict mutations of BAP1, PBRM1 and VHL genes. We then used TCIA-TCGA dataset to validate the predictive model build upon UTSW dataset. Results: The prediction accuracy of gene expression of TCIA-TCGA patients was 0.83 (20 of 24), 0.83 (20 of 24), and 0.75 (18 of 24) for BAP1, PBRM1, and VHL respectively. For BAP1 gene, texture feature was the most prominent feature type. For PBRM1 gene, intensity feature was the most prominent. For VHL gene, geometry, intensity, and texture features were all important. Conclusion: Using our feature selection strategy and models, we achieved predictive accuracy over 0.75 for all three genes under the condition of using patient data from one institution for training and data from other institutions for testing. These results suggest that radiogenomics can be used to aid in prognosis and used as convenient surrogates for expensive and time consuming gene assay procedures.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2015

Precision Higgs Boson Physics and Implications for Beyond the Standard Model Physics Theories

The discovery of the Higgs boson is one of science's most impressive recent achievements. We have taken a leap forward in understanding what is at the heart of elementary particle mass generation. We now have a significant opportunity to develop even deeper understanding of how the fundamental laws of nature are constructed. As such, we need intense focus from the scientific community to put this discovery in its proper context, to realign and narrow our understanding of viable theory based on this positive discovery, and to detail the implications the discovery has for theories that attempt to answer questions beyond what the Standard Model can explain. This project's first main object is to develop a state-of-the-art analysis of precision Higgs boson physics. This is to be done in the tradition of the electroweak precision measurements of the LEP/SLC era. Indeed, the electroweak precision studies of the past are necessary inputs to the full precision Higgs program. Calculations will be presented to the community of Higgs boson observables that detail just how well various couplings of the Higgs boson can be measured, and more. These will be carried out using state-of-the-art theory computations coupled with the new experimental results coming in from the LHC. The project's second main objective is to utilize the results obtained from LHC Higgs boson experiments and the precision analysis, along with the direct search studies at LHC, and discern viable theories of physics beyond the Standard Model that unify physics to a deeper level. Studies will be performed on supersymmetric theories, theories of extra spatial dimensions (and related theories, such as compositeness), and theories that contain hidden sector states uniquely accessible to the Higgs boson. In addition, if data becomes incompatible with the Standard Model's low-energy effective lagrangian, new physics theories will be developed that explain the anomaly and put it into a more unified framework beyond the Standard Model.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2013

Quality Research in Radiation Oncology Analysis of Clinical Performance Measures in the Management of Gastric Cancer

Background: The specific aim was to determine national patterns of radiation therapy (RT) practice in patients treated for stage IB-IV (nonmetastatic) gastric cancer (GC). Methods and Materials: A national process survey of randomly selected US RT facilities was conducted which retrospectively assessed demographics, staging, geographic region, practice setting, and treatment by using on-site record review of eligible GC cases treated from 2005 to 2007. Three clinical performance measures (CPMs), (1) use of computed tomography (CT)-based treatment planning; (2) use of dose volume histograms (DVHs) to evaluate RT dose to the kidneys and liver; and (3) completion of RT within the prescribed time frame; and emerging quality indicators, (i) use of intensity modulated RT (IMRT); (ii) use of image-guided tools (IGRT) other than CT for RT target delineation; and (iii) use of preoperative RT, were assessed. Results: CPMs were computed for 250 eligible patients at 45 institutions (median age, 62 years; 66% male; 60% Caucasian). Using 2000 American Joint Committee on Cancer criteria, 13% of patients were stage I, 29% were stage II, 32% were stage IIIA, 10% were stage IIIB, and 12% were stage IV. Most patients (43%) were treated at academic centers, 32% were treated at large nonacademic centers, and 25% were treated at small to medium sized facilities. Almost all patients (99.5%) underwent CT-based planning, and 75% had DVHs to evaluate normal tissue doses to the kidneys and liver. Seventy percent of patients completed RT within the prescribed time frame. IMRT and IGRT were used in 22% and 17% of patients, respectively. IGRT techniques included positron emission tomography (n=20), magnetic resonance imaging (n=1), respiratory gating and 4-dimensional CT (n=22), and on-board imaging (n=10). Nineteen percent of patients received preoperative RT. Conclusions: This analysis of radiation practice patterns for treating nonmetastatic GC indicates widespread adoption of CT-based planning with use of DVH to evaluate normal tissue doses. Most patients completed adjuvant RT in the prescribed time frame. IMRT and IGRT were not routinely incorporated into clinical practice during the 2005-2007 period. These data will be a benchmark for future Quality Research in Radiation Oncology GC surveys.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2013

Large Scale Parallel Solution of Incompressible Flow Problems Using Uintah and Hypre

The Uintah Software framework was developed to provide an environment for solving fluid-structure interaction problems on structured adaptive grids on large-scale, long-running, data-intensive problems. Uintah uses a combination of fluid-flow solvers and particle-based methods for solids together with a novel asynchronous task-based approach with fully automated load balancing. As Uintah is often used to solve incompressible flow problems in combustion applications it is important to have a scalable linear solver. While there are many such solvers available, the scalability of those codes varies greatly. The hypre software offers a range of solvers and pre-conditioners for different types of grids. The weak scalability of Uintah and hypre is addressed for particular examples of both packages when applied to a number of incompressible flow problems. After careful software engineering to reduce startup costs, much better than expected weak scalability is seen for up to 100K cores on NSFs Kraken architecture and up to260K cpu cores, on DOEs new Titan machine. The scalability is found to depend in a crtitical way on the choice of algorithm used by hypre for a realistic application problem.

Computer Science↗
DOE OSTI2021

Near L-edge Single and Multiple Photoionization of Doubly Charged Iron Ions

Using the photon-ion merged-beams technique at a synchrotron light source, we have measured relative cross sections for single and up to five-fold photoionization of Fe 2+ ions in the energy range of 690-920 eV. This range contains thresholds and resonances associated with ionization and excitation of 2p and 2s electrons. Calculations were performed to simulate the total absorption spectra. The theoretical results show very good agreement with the experimental data, if overall energy shifts of up to 2.5 eV are applied to the calculated resonance positions and assumptions are made about the initial experimental population of the various levels of the Fe 2+ ([Ar]3d 6 ) ground configuration. Furthermore, we performed extensive calculations of the Auger cascades that result when an electron is removed from the 2p subshell of Fe 2+ . These computations lead to a better agreement with the measured product-charge-state distributions as compared to earlier work. We conclude that the L-shell absorption features of low-charged iron ions are useful for identifying gas-phase iron in the interstellar medium and for discriminating against the various forms of condensed-phase iron bound to composite interstellar dust grains.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2015

CRISTAL V2.0: A New-generation criticality package - 14597

A new release of the CRISTAL French criticality calculation package, named CRISTAL V2.0, has been developed and intensively validated by IRSN and CEA, in collaboration with AREVA. This package was designed to cover the needs of criticality practitioners in the fields of nuclear fuel cycle facilities and transportation of fissile materials. Based on an optimized modular architecture in an updated software development environment, the design of CRISTAL package V2.0 integrates new generation of cross section libraries, new calculation schemes consistent with new applications, and up-to-date calculation codes capabilities. Four calculation routes are currently available allowing multi-group and continuous energy calculations. The validation database is composed of more than 3000 critical experiments, in order to cover a wide range of configurations. The CRISTAL V2.0 package also comprises customer services covering a range of areas of activity like documentation, hotline, maintenance and training courses. A wide promotion and an increased recognition referred both nationally and internationally are foreseen, including the provision of the CRISTAL V2.0 package to the OECD / NEA Data Bank. (authors)

42 ENGINEERING↗
DOE OSTI2022

Simulating Impacts of Extreme Events on Grids with High Penetrations of Wind Power Resources: Preprint

As extreme weather events become more frequent and intense, the demand for connecting grid operation and infrastructure planning with extreme event models will increase as well. We present a methodology for creating damage contingencies and scenarios for electric transmission grids during a hurricane strike. Included is an example case study: Hurricane Dolly damaging a synthetic 2000 bus test system during its landing in Southern Texas. Using WIND Toolkit meteorological data in conjunction with fragility curves for various electric grid elements, we generate stochastic damage scenarios that can be used for short- and long-term planning problems including emergency asset management. We perform statistical analysis of damages and quantify topological effects on example synthetic grid. Also, we investigate loss-of-load events during two-day economic dispatch experiment. Finally, we point out various shortcomings of our method and suggest how it can be improved.

contingencies↗
DOE OSTI2015

Evaluation of thermal helium beam and line-ratio fast diagnostic on the National Spherical Torus Experiment-Upgrade

A 1-D kinetic collisional radiative model (CRM) with state-of-the-art atomic data is developed and employed to simulate line emission to evaluate the Thermal Helium Beam (THB) diagnostic on NSTX-U. This diagnostic is currently in operation on RFX-mod, and it is proposed to be installed on NSTX-U. The THB system uses the intensity ratios of neutral helium lines 667.8, 706.5, and 728.1 nm to derive electron temperature (eV ) and density (cm –3 ) profiles. The purpose of the present analysis is to evaluate the applications of this diagnostic for determining fast (~4 μs) electron temperature and density radial profiles on the scrape-off layer (SOL) and edge regions of NSTX-U that are needed in turbulence studies. The diagnostic is limited by the level of detection of the 728.1 nm line, which is the weakest of the three. In conclusion, this study will also aid in future design of a similar 2-D diagnostic systems on the divertor.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Crowd cluster data in the USA for analysis of human response to COVID-19 events and policies

We provide data on daily social contact intensity of clusters of people at different types of Points of Interest (POI) by zip code in Florida and California. This data is obtained by aggregating fine-scaled details of interactions of people at the spatial resolution of 10 m, which is then normalized as a social contact index. We also provide the distribution of cluster sizes and average time spent in a cluster by POI type. This data will help researchers perform fine-scaled, privacy-preserving analysis of human interaction patterns to understand the drivers of the COVID-19 epidemic spread and mitigation. Current mobility datasets either provide coarse-level metrics of social distancing, such as radius of gyration at the county or province level, or traffic at a finer scale, neither of which is a direct measure of contacts between people. We use anonymized, de-identified, and privacy-enhanced location-based services (LBS) data from opted-in cell phone apps, suitably reweighted to correct for geographic heterogeneities, and identify clusters of people at non-sensitive public areas to estimate fine-scaled contacts.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2014

Community-driven development for computational biology at Sprints, Hackathons and Codefests

Background: Computational biology comprises a wide range of technologies and approaches. Multiple technologies can be combined to create more powerful workflows if the individuals contributing the data or providing tools for its interpretation can find mutual understanding and consensus. Much conversation and joint investigation are required in order to identify and implement the best approaches. Traditionally, scientific conferences feature talks presenting novel technologies or insights, followed up by informal discussions during coffee breaks. In multi-institution collaborations, in order to reach agreement on implementation details or to transfer deeper insights in a technology and practical skills, a representative of one group typically visits the other. However, this does not scale well when the number of technologies or research groups is large. Conferences have responded to this issue by introducing Birds-of-a-Feather (BoF) sessions, which offer an opportunity for individuals with common interests to intensify their interaction. However, parallel BoF sessions often make it hard for participants to join multiple BoFs and find common ground between the different technologies, and BoFs are generally too short to allow time for participants to program together. Results: This report summarises our experience with computational biology Codefests, Hackathons and Sprints, which are interactive developer meetings. They are structured to reduce the limitations of traditional scientific meetings described above by strengthening the interaction among peers and letting the participants determine the schedule and topics. These meetings are commonly run as loosely scheduled “unconferences” (self-organized identification of participants and topics for meetings) over at least two days, with early introductory talks to welcome and organize contributors, followed by intensive collaborative coding sessions. We summarise some prominent achievements of those meetings and describe differences in how these are organised, how their audience is addressed, and their outreach to their respective communities. Conclusions: Hackathons, Codefests and Sprints share a stimulating atmosphere that encourages participants to jointly brainstorm and tackle problems of shared interest in a self-driven proactive environment, as well as providing an opportunity for new participants to get involved in collaborative projects.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2015

Final Technical Report for Interagency Agreement No. DE-SC0005453 “Characterizing Aerosol Distributions, Types, and Optical and Microphysical Properties using the NASA Airborne High Spectral Resolution Lidar (HSRL) and the Research Scanning Polarimeter (RSP)”

Measurements of the vertical profile of atmospheric aerosols and aerosol optical and microphysical characteristics are required to: 1) determine aerosol direct and indirect radiative forcing, 2) compute radiative flux and heating rate profiles, 3) assess model simulations of aerosol distributions and types, and 4) establish the ability of surface and space-based remote sensors to measure the indirect effect. Consequently the ASR program calls for a combination of remote sensing and in situ measurements to determine aerosol properties and aerosol influences on clouds and radiation. As part of our previous DOE ASP project, we deployed the NASA Langley airborne High Spectral Resolution Lidar (HSRL) on the NASA B200 King Air aircraft during major field experiments in 2006 (MILAGRO and MaxTEX), 2007 (CHAPS), 2009 (RACORO), and 2010 (CalNex and CARES). The HSRL provided measurements of aerosol extinction (532 nm), backscatter (532 and 1064 nm), and depolarization (532 and 1064 nm). These measurements were typically made in close temporal and spatial coincidence with measurements made from DOE-funded and other participating aircraft and ground sites. On the RACORO, CARES, and CalNEX missions, we also deployed the NASA Goddard Institute for Space Studies (GISS) Research Scanning Polarimeter (RSP). RSP provided intensity and degree of linear polarization over a broad spectral and angular range enabling column-average retrievals of aerosol optical and microphysical properties. Under this project, we analyzed observations and model results from RACORO, CARES, and CalNex and accomplished the following objectives. 1. Identified aerosol types, characterize the vertical distribution of the aerosol types, and partition aerosol optical depth by type, for CARES and CalNex using HSRL data as we have done for previous missions. 2. Investigated aerosol microphysical and macrophysical properties using the RSP. 3. Used the aerosol backscatter and extinction profiles measured by the HSRL to characterize the planetary boundary layer height (PBL) and the transition zone thickness, for the RACORO and CARES and CalNex campaigns as we have done for previous campaigns. 4. Investigated how optical properties measured by HSRL vary near clouds. 5. Assessed model simulations of aerosol spatial distributions and optical and microphysical properties.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2014

Temperature elevation by HIFU in ex vivo porcine muscle: MRI measurement and simulation study

Purpose: High-intensity focused ultrasound is a rapidly developing medical technology with a large number of potential clinical applications. Computational model can play a pivotal role in the planning and optimization of the treatment based on the patient's image. Nonlinear propagation effects can significantly affect the temperature elevation and should be taken into account. In order to investigate the importance of nonlinear propagation effects, nonlinear Westervelt equation was solved. Weak nonlinear propagation effects were studied. The purpose of this study was to investigate the correlation between the predicted and measured temperature elevations and lesion in a porcine muscle. Methods: The investigated single-element transducer has a focal length of 12 cm, an aperture of 8 cm, and frequency of 1.08 MHz. Porcine muscle was heated for 30 s by focused ultrasound transducer with an acoustic power in the range of 24–56 W. The theoretical model consists of nonlinear Westervelt equation with relaxation effects being taken into account and Pennes bioheat equation. Results: Excellent agreement between the measured and simulated temperature rises was found. For peak temperatures above 85–90 °C “preboiling” or cavitation activity appears and lesion distortion starts, causing small discrepancy between the measured and simulated temperature rises. From the measurements and simulations, it was shown that distortion of the lesion was caused by the “preboiling” activity. Conclusions: The present study demonstrated that for peak temperatures below 85–90 °C numerical simulation results are in excellent agreement with the experimental data in three dimensions. Both temperature rise and lesion size can be well predicted. Due to nonlinear effect the temperature in the focal region can be increased compared with the linear case. The current magnetic resonance imaging (MRI) resolution is not sufficient. Due to the inevitable averaging the measured temperature can be 10–30 °C lower than the peak temperature. Computational fluid dynamics can provide additional important information that is lost using a state of the art MRI device.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2024

Review and meta-analysis of electron temperatures from high-intensity laser–solid interactions

The accelerated electron spectrum from high-intensity laser–solid interaction is often conveniently described using a Boltzmann distribution, whose temperature is known within the field as the hot-electron temperature. The importance of the electron temperature is highlighted by the sheer number of experimental and simulation studies on the subject over the past three decades. Recently, multi-kJ, multi-ps pulses have yielded electron spectra with temperatures far beyond the expected ponderomotive result. Expressions that predict the electron temperature considering laser parameters beyond intensity and wavelength have been developed, albeit using small datasets. In this review, we present what is, to the best of our knowledge, the largest dataset of electron temperatures gathered from experimental measurements and particle-in-cell simulations. This dataset allows us to compare existing analytical and empirical hot-electron temperature scaling models over a wide parameter range. We also develop new scaling models that incorporate the laser pulse duration of the laser and the plasma scale length. Three models that include pulse-duration and scale length dependence are especially successful at predicting both simulated and experimental data. The dataset will soon be made publicly available to encourage further investigation.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

A bi-level spatiotemporal clustering approach and its application to drought extraction

We present a novel flexible bi-level spatiotemporal clustering algorithm to extract events based on their intensity and spatiotemporal structures. Our algorithm consists of using (i) a novel space-time k-means clustering to obtain spatiotemporally coherent intensity clusters, and (ii) a density-based spatial clustering of applications with noise (DBSCAN) to spatiotemporally section the intensity clusters into individual events. We discuss the development of the algorithm, the selection, tuning and meaning of the parameters within each step, as well as its validation. Finally, we apply the algorithm to a spatiotemporal drought index, standardized vapor pressure deficit drought index (SVDI), over the continental United States (US) from 1980–2021 and show that it captures historical drought events over the continental United States and their spatiotemporal extents.

17 WIND ENERGY↗
DOE OSTI2022

Simulating Impacts of Extreme Events on Grids with High Penetrations of Wind Power Resources

As extreme weather events become more frequent and intense, the demand for connecting grid operation and infrastructure planning with extreme event models will increase as well. We present a methodology for creating damage contingencies and scenarios for electric transmission grids during a hurricane strike. Included is an example case study: Hurricane Dolly damaging a synthetic 2000 bus test system during its landing in Southern Texas. Using WIND Toolkit meteorological data in conjunction with fragility curves for various electric grid elements, we generate stochastic damage scenarios that can be used for short- and long-term planning problems including emergency asset management. We perform statistical analysis of damages and quantify topological effects on example synthetic grid. Also, we investigate loss-of-load events during two-day economic dispatch experiment. Finally, we point out various shortcomings of our method and suggest how it can be improved.

data-driven forecasting↗
DOE OSTI2023

A National Infrastructure for Artificial Intelligence on the Grid (NI4AI) (Final Scientific/Technical Report)

Electric utilities have traditionally taken a very pragmatic yet myopic approach with grid sensors and the resulting collected data. Sensors are purchased and deployed to solve a specific, known problem that has risen to sufficient awareness as to justify the effort of deploying sensors and the needed capital investment. This sensor data flows into proprietary software packages with limited functionality intended only to address the initial problem. This approach aligns with the financial incentives of the utility to deploy capital into fixed hardware assets for which the corporations earn a rate of return. This mentality stands in stark contrast to the big data revolution that started nearly 25 years ago with the rise of Google. In this worldview, data is a fundamental business asset; successful organizations collect, store, explore, merge, and exploit as much data as possible to not only solve problems well understood today but also to tackle new problems that will inevitably rise tomorrow. The ARPA-E Open Innovation 2018 project entitled A National Infrastructure for Artificial Intelligence on the Grid or NI4AI for short was designed to demonstrate this alternative paradigm for using data. To do this, the project was composed of three key thrust areas. The first major component deployed a variety of high-frequency grid sensors and captured terabytes of both wide-scale and localized grid measurements, generating high-value datasets for grid research and algorithm development. The second aspect made available PingThings’ PredictiveGridTM, a horizontally scalable, cloud-based data management and AI platform built for time series data to explore and exploit the collected data. Finally, the project fostered a diverse and open research community composed of experts from numerous fields through focused educational content, code sharing, and data science competitions. Shifting away from “single use” sensors and closed data silos within electric utilities is a major benefit to the public at large. This legacy approach to data is incredibly (1) capital intensive (new sensors must be deployed for each new problem and problems tend to arise continuously) and (2) painfully slow (new problems must be identified first and then new sensors must be deployed to collect data to begin to address the issue). The transition to a carbon neutral grid requires a massive transformation of the existing grid infrastructure and will continue to challenge the legacy grid in unforeseen ways. The only way to make the energy transition cost effective is for utilities to abandon this dated data paradigm and adopt more contemporary approaches. NI4AI has shown that it is technically possible and economically feasible to ingest, explore, and exploit grid data collected from even very high frequency sensing, such as continuous point on wave sensors collecting measurements 10,000 times a second. In fact, the PredictiveGrid platform used is commercially available and deployed at several utilities in the United States. Project accomplishments were numerous and included (1) making available a state of the art time series platform to the community, (2) collecting over 520 streams of time series data from grid sensors totaling over 1 trillion grid measurements, and (3) developing and nurturing a community within the industry focused on the use of data to create value for utilities and, ultimately, end consumers.

97 MATHEMATICS AND COMPUTING↗