Search NASA⌕ Search

SEARCH · Search NASA

Results for “hyperspectral data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Chlorophyll- a dynamics in the lower Amazon River: insights from in situ and hyperspectral remote sensing using OCI-PACE

Chlorophyll-a concentration (Chla) is a key indicator of phytoplankton biomass and aquatic trophic status. However, satellite-derived Chla in sediment-rich waters, such as those found in the Lower Amazon River, remains challenging. The present study characterizes in situ Chla levels and their relationships with geographic, physical, and biogeochemical parameters in the Lower Amazon. Data collected between 2014 and 2017 across four hydrological seasons included measurements of Chla, remote sensing reflectance, and water quality parameters such as total suspended sediment, conductivity, water surface temperature, dissolved oxygen, pH, dissolved organic carbon and coloured dissolved organic matter. An empirical model was developed to estimate Chla using simulated hyperspectral bands from NASA’s PACE mission, achieving high performance (R 2 = 0.76; RMSE = 0.11 μg·L −1 ). Red bands proved particularly effective for Chla retrieval, while the addition of ultraviolet bands further enhanced model accuracy. The application of the developed model to satellite imagery yielded results consistent with in situ observations for the same hydrologic season. Seasonal variation and geographic location were major factors influencing Chla dynamics. This study provides a novel contribution to Chla estimation in optically complex, highly turbid waters and highlights the potential of the PACE mission to enhance global aquatic ecosystem monitoring. In conclusion, by offering freely available hyperspectral data with high radiometric resolution, PACE represents a significant advancement in the realm of remote sensing of aquatic environments.

Amazon River↗
NASA NTRS1998

Support Vector Machines for Hyperspectral Remote Sensing Classification

The Support Vector Machine provides a new way to design classification algorithms which learn from examples (supervised learning) and generalize when applied to new data. We demonstrate its success on a difficult classification problem from hyperspectral remote sensing, where we obtain performances of 96%, and 87% correct for a 4 class problem, and a 16 class problem respectively. These results are somewhat better than other recent results on the same data. A key feature of this classifier is its ability to use high-dimensional data without the usual recourse to a feature selection step to reduce the dimensionality of the data. For this application, this is important, as hyperspectral data consists of several hundred contiguous spectral channels for each exemplar. We provide an introduction to this new approach, and demonstrate its application to classification of an agriculture scene.

Gualtieri, J. Anthony↗
NASA NTRS1994

(abstract) Application of Neural Networks to Hyperspectral Unmixing

The emergence, in recent years, of hyperspectral sensors provides a tremendous opportunity for advancing the process of detailed and direct remote detection and identification from space of targets or surface materials. Such sensors exploit the uniqueness of the corresponding spectral reflectance signatures, which enables high resolution imaging spectrometer data to be processed on a pixel-by-pixel basis. This has implications both for defense-related applications (e.g., surveillance tasks) and in the civilian domain (e.g., for science applications). The purpose of this talk is to discuss a number of strong arguments that support neural networks as a choice for the generalized analysis (e.g., unmixing) of remotely sensed hyperspectral data.

signatures moving object detection surveillance co↗
DOE OSTI2023

Towards operational atmospheric correction of airborne hyperspectral imaging spectroscopy: Algorithm evaluation, key parameter analysis, and machine learning emulators

Atmospheric correction of airborne hyperspectral imaging spectroscopy (AHIS) to obtain high-quality surface reflectance is the prerequisite for remote sensing applications. Over the last decades, different atmospheric correction methods have been developed based on radiative transfer models (RTMs), however, the relative performances of different algorithms are unclear. Automated operational atmospheric correction methods to process large-volume AHIS data in a high-accurate and high-throughput manner are still lacking. Therefore, this study proposed an operational atmospheric correction pipeline for deriving surface reflectance from AHIS data. To ensure the accuracy and efficiency of the pipeline, we focused on three specific aspects: (1) selecting a suitable RTM for the development of atmospheric lookup tables (LUTs) by comparing the commercial MODerate resolution atmospheric TRANsmission (MODTRAN) and open-sourced Library for Radiative TRANsfer (LibRadTRAN) models, where the widely-used software, Atmospheric/Topographic Correction for Airborne Imagery (ATCOR), was used as benchmarks; (2) identifying key atmospheric correction parameters and determining suitable sources for parameter retrievals including AHIS, Moderate Resolution Imaging Spectroradiometer (MODIS), and AErosol RObotic NETwork (AERONET); and (3) testing the performance of using machine learning emulators to speed up the RTM-based atmospheric correction. Results indicate that (1) atmospheric correction based on MODTRAN LUTs can produce surface reflectance accurately with mean absolute errors < 0.05 and cosine similarities > 0.98 compared to field measurements, which is comparable to the software ATCOR and slightly outperforms the LibRadTRAN LUTs; (2) sobol global sensitivity analysis demonstrates that in the atmospheric correction, visibility and water vapor are two key parameters that can be accurately derived from AHIS in contrast to MODIS or AERONET data; and (3) Random Forest emulators can produce accurate estimations of surface reflectance with mean absolute errors < 0.03 and cosine similarities > 0.98 for higher processing efficiency and determine a suitable set of wavelengths for retrieving atmospheric visibility and water vapor. In conclusion, the proposed atmospheric correction pipeline also improved the four-stream radiative transfer theory for airborne applications by considering adjacent effects from airborne surrounding pixels and can also be applied for atmospheric correction of hyperspectral data from spaceborne missions.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Grass Evolutionary Lineages Can Be Identified Using Hyperspectral Leaf Reflectance

Hyperspectral remote sensing has the potential to map numerous attributes of the Earth’s surface, including spatial patterns of biological diversity. Grasslands are one of the largest biomes on Earth. Accurate mapping of grassland biodiversity relies on spectral discrimination of endmembers of species or plant functional types. We focused on spectral separation of grass lineages that dominate global grassy biomes: Andropogoneae (C 4 ), Chloridoideae (C 4 ), and Pooideae (C 3 ). We examined leaf reflectance spectra (350–2,500 nm) from 43 grass species representing these grass lineages from four representative grassland sites in the Great Plains region of North America. Here, we assessed the utility of leaf reflectance data for classification of grass species into three major lineages and by collection site. Classifications had very high accuracy (94%) that were robust to site differences in species and environment. We also show an information loss using multispectral sensors, that is, classification accuracy of grass lineages using spectral bands provided by current multispectral satellites is much lower (accuracy of 85.2% and 61.3% using Sentinel 2 and Landsat 8 bands, respectively). Our results suggest that hyperspectral data have an exciting potential for mapping grass functional types as informed by phylogeny. Leaf-level hyperspectral separability of grass lineages is consistent with the potential increase in biodiversity and functional information content from the next generation of satellite-based spectrometers.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2009

Sparse Superpixel Unmixing for Exploratory Analysis of CRISM Hyperspectral Images

Fast automated analysis of hyperspectral imagery can inform observation planning and tactical decisions during planetary exploration. Products such as mineralogical maps can focus analysts' attention on areas of interest and assist data mining in large hyperspectral catalogs. In this work, sparse spectral unmixing drafts mineral abundance maps with Compact Reconnaissance Imaging Spectrometer (CRISM) images from the Mars Reconnaissance Orbiter. We demonstrate a novel "superpixel" segmentation strategy enabling efficient unmixing in an interactive session. Tests correlate automatic unmixing results based on redundant spectral libraries against hand-tuned summary products currently in use by CRISM researchers.

Image Segmentation↗
NASA NTRS2001

Nutrient Stress Detection in Corn Using Neural Networks and AVIRIS Hyperspectral Imagery

AVIRIS image cube data has been processed for the detection of nutrient stress in corn by both known, ratio-type algorithms and by trained neural networks. The USDA Shelton, NE, ARS Variable Rate Nitrogen Application (VRAT) experimental farm was the site used in the study. Upon application of ANOVA and Dunnett multiple comparsion tests on the outcome of both the neural network processing and the ratio-type algorithm results, it was found that the neural network methodology provides a better overall capability to separate nutrient stressed crops from in-field controls.

Estep, Lee↗
DOE OSTI2022

Airborne hyperspectral imaging of cover crops through radiative transfer process-guided machine learning

Cover cropping between cash crop growing seasons is a multifunctional conservation practice. Timely and accurate monitoring of cover crop traits, notably aboveground biomass and nutrient content, is beneficial to agricultural stakeholders to improve management and understand outcomes. Currently, there is a scarcity of spatially and temporally resolved information for assessing cover crop growth. Remote sensing has a high potential to fill this need, but conventional empirical regression operated with coarse-resolution multispectral data has large uncertainties. Therefore, this study utilized airborne hyperspectral imaging techniques and developed new process-guided machine learning approaches (PGML) for cover crop monitoring. Specifically, we deployed an airborne hyperspectral system covering visible to shortwave-infrared wavelengths (400–2400 nm) to acquire high spatial (0.5 m) and spectral (3–5 nm) resolution reflectance over 23 cover crop fields across Central Illinois in March and April of 2021. Airborne hyperspectral surface reflectance with high spectral and spatial resolution can be well matched with field data to quantify cover crop traits. Furthermore, the PGML models were pre-trained by synthetic data from soil-vegetation radiative transfer modeling (one million records), and then fine-tuned with field data of cover crop biomass and nutrient content. Results show that airborne hyperspectral data with PGML can achieve high accuracy to predict cover crop aboveground biomass (R 2 = 0.72, relative RMSE = 15.16%) and nitrogen content (R 2 = 0.69, relative RMSE = 16.59%) through leave-one-field-out cross-validation. Unlike the pure data-driven approach (e.g., partial least-squares regression), PGML incorporated radiative transfer knowledge and obtained higher predictive performance with fewer field data. Meanwhile, with field data for model fine-tuning, PGML predicted biomass more accurately than the inversion of radiative transfer models. Here we also found that the red edge has a high contribution in quantifying aboveground biomass and nitrogen content, followed by green and shortwave spectra. This study demonstrated the first attempt of utilizing hyperspectral remote sensing to accurately quantify cover crop traits. We highlight the strength of PGML in exploiting sensing data to quantify ecosystem variables to advance agroecosystem monitoring for sustainable agricultural management.

60 APPLIED LIFE SCIENCES↗
NASA NTRS2015

Double-Diffusive Convection During Growth of Halides and Selenides

Heavy metal halides and selenides have unique properties which make them excellent materials for chemical, biological and radiological sensors. Recently it has been shown that selenohalides are even better materials than halides or selenides for gamma-ray detection. These materials also meet the strong needs of a wide band imaging technology to cover ultra-violet (UV), midwave infrared wavelength (MWIR) to very long wavelength infrared (VLWIR) region for hyperspectral imager components such as etalon filters and acousto-optic tunable filters (AO). In fact AOTF based imagers based on these materials have some superiority than imagers based on liquid crystals, FTIR, Fabry-Perot, grating, etalon, electro-optic modulation, piezoelectric and several other concepts. For example, broadband spectral and imagers have problems of processing large amount of information during real-time observation. Acousto-Optic Tunable Filter (AOTF) imagers are being developed to fill the need of reducing processing time of data, low cost operation and key to achieving the goal of covering long-wave infrared (LWIR). At the present time spectral imaging systems are based on the use of diffraction gratings are typically used in a pushbroom or whiskbroom mode. They are mostly used in systems and acquire large amounts of hyperspectral data that is processed off-line later. In contrast, acousto-optic tunable filter spectral imagers require very little image processing, providing new strategies for object recognition and tracking. They are ideally suited for tactical situations requiring immediate real-time image processing. But the performance of these imagers depends on the quality and homogeneity of acousto-optic materials. In addition for many systems requirements are so demanding that crystals up to sizes of 10 cm length are desired. We have studied several selenides and halide crystals for laser and AO imagers for MWIR and LWIR wavelength regions. We have grown and fabricated crystals of several materials such as mercurous chloride, mercurous bromide, mercurous iodide, lead chloride lead bromide, lead iodide, thallium arsenic selenide, gallium selenide, zince sulfide zinc selenide and several crystals into devices. We have used both Bridgman and physical vapor transport (PVT) crystal growth methods. In the past have examined PVT growth numerically for conditions where the boundary of the enclosure is subjected to a nonlinear thermal profile. Since past few months we have been working on binary and ternary materials such as selenoiodides, doped zinc sulfides and mercurous chloro bromide and mercurous bromoiodides. In the doped and ternary materials thermal and solutal convection play extremely important role during the growth. Very commonly striations and banding is observed. Our experiments have indicated that even in highly purified source materials, homogeneity in 1-g environment is very difficult. Some of our previous numerical studies have indicated that gravity level less than 10-4 (-g) helps in controlling the thermosolutal convection. We will discuss the ground based growth results of HgClxBr(1-x) and ZnSe growth results for the mm thick to large cm size crystals. These results will be compared with our microgravity experiments performed with this class of materials. For both HgCl-HgBr and ZnS-ZnSe the lattice parameters of the mixtures obey Vagard's law in the studied composition range. The study demonstrates that properties are very anisotropic with crystal orientation, and performance achievement requires extremely careful fabrication to utilize highest figure of merit. In addition, some parameters such as crystal growth fabrication, processing time, resolution, field of view and efficiency will be described based on novel solid solution materials. It was predicted that very similar to the pure compounds solid solutions also have very large anisotropy, and very precise oriented and homogeneous bulk and thin film crystals is required to achieve maximum performance of laser or imagers. Some of the parameters controlling the homogeneity such as thermos-solutal convection driven forces can be controlled in microgravity environments to utilize the benefits of these unique materials.

Singh, N. B.↗
NASA NTRSDate not supplied

Overview of the NASA Earth Action Strategies Wildland Fire Initiative

As part of NASA’s new Earth Action strategy, the Wildland Fire initiative was established, which includes both the NASA Wildland Fire Program (WFP) and the FireSense project. NASA has over 50 years of experience generating data and technology to enhance fire science and operational management. The WFP’s mission is threefold: 1) assemble communities of practice through collaborative efforts with government, academia, and the private sector; 2) co-develop knowledge and applications with relevant partners and stakeholders in the wildfire community; and 3) improve wildland fire management through the transitioning of NASA data, technology, tools, and science to stakeholder organizations. The WFP is focusing on supporting proactive fire management, including situational awareness, preparedness, and risk mitigation. This will be accomplished through selected projects that identify management challenges, relevant to partners and end users, and the NASA data that will be utilized to deliver innovative solutions to enhance the management of wildland fires. Examples include: i) investigation of evaporative stress from OpenET to help predict the risk of wildfire occurrence in watersheds; ii) incorporation of space based LiDAR for the generation of 3-dimensional forest fuel metrics, used to improve wildfire risk and behavior models; iii) integration of global, multi-platform geostationary active-fire data in near-real-time into NASA’s Fire Information for Resource Management System (FIRMS); and iv) identification of post-fire ecohydrological conditions using thermal, multispectral, synthetic aperture radar (SAR), and hyperspectral remotely-sensed data to improve flood hazard forecast models. The FireSense project is a US-focused 5-year project that will focus on delivering NASA’s unique Earth science and technological capabilities to operational agencies, striving towards enhancing fire fighting and air quality management. The project will include airborne campaigns and new technology that will likely have global implications. Initial stakeholder engagement led FireSense to focus on four use-cases focused on the characterization and measurement of: (i) pre-fire fuels conditions, (ii) active fire-dynamics; (iii) post-fire impact and threats; and iv) air quality impacts and forecasting, each-developed with identified stakeholders.

Wildland Fire program↗
DOE OSTI2025

Uncovering multiscale structure-property correlations via active learning in scanning tunneling microscopy

Atomic arrangements and local sub-structures fundamentally influence emergent material functionalities. These structures are conventionally probed using spatially resolved studies and the property correlations are deciphered by a researcher based on sequential explorations, thereby limiting the efficiency and scope. Here we demonstrate a multi-scale Bayesian deep-learning based framework that automatically correlates material structure with its electronic properties using scanning tunneling microscopy (STM) measurements in real-time. Its predictions are used to autonomously direct exploration toward regions of the sample that optimize a given material property. This method is deployed on a low-temperature ultra-high vacuum STM to understand the structure-property relationship in a europium-based semimetal, EuZn 2 As 2 , a promising candidate relevant to magnetism-driven topological phenomena. The framework employs a sparse-sampling approach to efficiently construct the scalar-property space using minimal measurements, about 1–10% of the data required in standard hyperspectral methods. Moreover, we formulate the problem hierarchically across length scales, implementing autonomous workflow to locate mesoscopic and atomic structures that correspond to a target material property. This framework offers the choice to design scalar-property from the spectroscopic data to steer sample exploration. Our findings reveal correlations of the electronic properties unique to surface terminations, local defect density, and point defects.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
NASA NTRS2018

NeMO-Net & Fluid Lensing: The Neural Multi-Modal Observation & Training Network for Global Coral Reef Assessment Using Fluid Lensing Augmentation of NASA EOS Data

We present preliminary results from NASA NeMO-Net, the first neural multi-modal observation and training network for global coral reef assessment. NeMO-Net is an open-source deep convolutional neural network (CNN) and interactive active learning training software in development which will assess the present and past dynamics of coral reef ecosystems. NeMO-Net exploits active learning and data fusion of mm-scale remotely sensed 3D images of coral reefs captured using fluid lensing with the NASA FluidCam instrument, presently the highest-resolution remote sensing benthic imaging technology capable of removing ocean wave distortion, as well as hyperspectral airborne remote sensing data from the ongoing NASA CORAL mission and lower-resolution satellite data to determine coral reef ecosystem makeup globally at unprecedented spatial and temporal scales. Aquatic ecosystems, particularly coral reefs, remain quantitatively misrepresented by low-resolution remote sensing as a result of refractive distortion from ocean waves, optical attenuation, and remoteness. Machine learning classification of coral reefs using FluidCam mm-scale 3D data show that present satellite and airborne remote sensing techniques poorly characterize coral reef percent living cover, morphology type, and species breakdown at the mm, cm, and meter scales. Indeed, current global assessments of coral reef cover and morphology classification based on km-scale satellite data alone can suffer from segmentation errors greater than 40%, capable of change detection only on yearly temporal scales and decameter spatial scales, significantly hindering our understanding of patterns and processes in marine biodiversity at a time when these ecosystems are experiencing unprecedented anthropogenic pressures, ocean acidification, and sea surface temperature rise. NeMO-Net leverages our augmented machine learning algorithm that demonstrates data fusion of regional FluidCam (mm, cm-scale) airborne remote sensing with global low-resolution (m, km-scale) airborne and spaceborne imagery to reduce classification errors up to 80% over regional scales. Such technologies can substantially enhance our ability to assess coral reef ecosystems dynamics.

satellite data↗
NASA NTRS2018

NASA NeMO-Net

We present preliminary results from NASA NeMO-Net, the first neural multi-modal observation and training network for global coral reef assessment. NeMO-Net is an open-source deep convolutional neural network (CNN) and interactive active learning training software in development which will assess the present and past dynamics of coral reef ecosystems. NeMO-Net exploits active learning and data fusion of mm-scale remotely sensed 3D images of coral reefs captured using fluid lensing with the NASA FluidCam instrument, presently the highest-resolution remote sensing benthic imaging technology capable of removing ocean wave distortion, as well as hyperspectral airborne remote sensing data from the ongoing NASA CORAL mission and lower-resolution satellite data to determine coral reef ecosystem makeup globally at unprecedented spatial and temporal scales. Aquatic ecosystems, particularly coral reefs, remain quantitatively misrepresented by low- resolution remote sensing as a result of refractive distortion from ocean waves, optical attenuation, and remoteness. Machine learning classification of coral reefs using FluidCam mm-scale 3D data show that present satellite and airborne remote sensing techniques poorly characterize coral reef percent living cover, morphology type, and species breakdown at the mm, cm, and meter scales. Indeed, current global assessments of coral reef cover and morphology classification based on km-scale satellite data alone can suffer from segmentation errors greater than 40%, capable of change detection only on yearly temporal scales and decameter spatial scales, significantly hindering our understanding of patterns and processes in marine biodiversity at a time when these ecosystems are experiencing unprecedented anthropogenic pressures, ocean acidification, and sea surface temperature rise. NeMO-Net leverages our augmented machine learning algorithm that demonstrates data fusion of regional FluidCam (mm, cm-scale) airborne remote sensing with global low-resolution (m, km-scale) airborne and spaceborne imagery to reduce classification errors up to 80% over regional scales. Such technologies can substantially enhance our ability to assess coral reef ecosystems dynamics.

NASA↗
NASA NTRS2008

Processing LiDAR Data to Predict Natural Hazards

ELF-Base and ELF-Hazards (wherein 'ELF' signifies 'Extract LiDAR Features' and 'LiDAR' signifies 'light detection and ranging') are developmental software modules for processing remote-sensing LiDAR data to identify past natural hazards (principally, landslides) and predict future ones. ELF-Base processes raw LiDAR data, including LiDAR intensity data that are often ignored in other software, to create digital terrain models (DTMs) and digital feature models (DFMs) with sub-meter accuracy. ELF-Hazards fuses raw LiDAR data, data from multispectral and hyperspectral optical images, and DTMs and DFMs generated by ELF-Base to generate hazard risk maps. Advanced algorithms in these software modules include line-enhancement and edge-detection algorithms, surface-characterization algorithms, and algorithms that implement innovative data-fusion techniques. The line-extraction and edge-detection algorithms enable users to locate such features as faults and landslide headwall scarps. Also implemented in this software are improved methodologies for identification and mapping of past landslide events by use of (1) accurate, ELF-derived surface characterizations and (2) three LiDAR/optical-data-fusion techniques: post-classification data fusion, maximum-likelihood estimation modeling, and hierarchical within-class discrimination. This software is expected to enable faster, more accurate forecasting of natural hazards than has previously been possible.

Fairweather, Ian↗
NASA NTRS1999

High Data Rate Instrument Study

The High Data Rate Instrument Study was a joint effort between the Jet Propulsion Laboratory (JPL) and the Goddard Space Flight Center (GSFC). The objectives were to assess the characteristics of future high data rate Earth observing science instruments and then to assess the feasibility of developing data processing systems and communications systems required to meet those data rates. Instruments and technology were assessed for technology readiness dates of 2000, 2003, and 2006. The highest data rate instruments are hyperspectral and synthetic aperture radar instruments which are capable of generating 3.2 Gigabits per second (Gbps) and 1.3 Gbps, respectively, with a technology readiness date of 2003. These instruments would require storage of 16.2 Terebits (Tb) of information (RF communications case of two orbits of data) or 40.5 Tb of information (optical communications case of five orbits of data) with a technology readiness date of 2003. Onboard storage capability in 2003 is estimated at 4 Tb; therefore, all the data created cannot be stored without processing or compression. Of the 4 Tb of stored data, RF communications can only send about one third of the data to the ground, while optical communications is estimated at 6.4 Tb across all three technology readiness dates of 2000, 2003, and 2006 which were used in the study. The study includes analysis of the onboard processing and communications technologies at these three dates and potential systems to meet the high data rate requirements. In the 2003 case, 7.8% of the data can be stored and downlinked by RF communications while 10% of the data can be stored and downlinked with optical communications. The study conclusion is that only 1 to 10% of the data generated by high data rate instruments will be sent to the ground from now through 2006 unless revolutionary changes in spacecraft design and operations such as intelligent data extraction are developed.

Schober, Wayne↗
DOE OSTI2021

Randomized Algorithms for Scientific Computing (RASC)

Randomized algorithms have propelled advances in artificial intelligence (AI) and represent a foundational research area in advancing AI for Science. Future advancements in DOE Office of Science priority areas such as climate science, astrophysics, fusion, advanced materials, combustion, and quantum computing all require randomized algorithms for surmounting challenges of complexity, robustness, and scalability. Advances in data collection and numerical simulation have changed the dynamics of scientific research and motivate the need for randomized algorithms. For instance, advances in imaging technologies such as X-ray ptychography, electron microscopy, electron energy loss spectroscopy, or adaptive optics lattice light-sheet microscopy collect hyperspectral imaging and scattering data in terabytes, at breakneck speed enabled by state-of-the-art detectors. The data collection is exceptionally fast compared with its analysis. Likewise, advances in high-performance architectures have made exascale computing a reality and changed the economies of scientific computing in the process. Floating-point operations that create data are essentially free in comparison with data movement. Thus far, most approaches have focused on creating faster hardware. Ironically, this faster hardware has exacerbated the problem by making data still easier to create. Under such an onslaught, scientists often resort to heuristic deterministic sampling schemes (e.g., low-precision arithmetic, sampling every nth element) and sacrifice potentially valuable accuracy. Dramatically better results can be achieved via randomized algorithms, reducing the data size as much as or more than naive deterministic subsampling can achieve, while retaining the high accuracy of computing on the full data set. By randomized algorithms we mean those algorithms that employ some form of randomness in internal algorithmic decisions to accelerate time to solution, increase scalability, or improve reliability. Examples include matrix sketching for solving large-scale least-squares problems (see Figure 1) and stochastic gradient descent for training machine learning models. We are not recommending heuristic methods but rather randomized algorithms that have certificates of correctness and probabilistic guarantees of optimality and near-optimality. Such approaches can be useful beyond acceleration, for example, in understanding how to avoid measure zero worst-case scenarios that plague methods such as QR matrix factorization.

97 MATHEMATICS AND COMPUTING↗