Search NASASearch

SEARCH · Search NASA

Results for “Data frames”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Raw_data_Batch_I: Garfield Ridge

Date of collection: May 4, 2023 Location: Garfield Ridge, Chicago, IL This data set contains lidar and vision data collected in Garfield Ridge, Chicago. The vehicle started from the intersection of Garfield Ridge and S. Harlem, headed east until S. Central Ave. The vehicle headed south along S. Central Ave. until West 60th Street, headed west, and turned north along S. Austin Ave. until it turned west onto W. 59th Street. The vehicle then headed north along S. Harlem Ave. and returned to the intersection of Garfield Ridge and S. Harlem. The data contains the following Robot Operating System (ROS) topics: - /Central_Camera_blurred – Image flow from coaxial camera heading, vehicle front. - /Left_Camera_blurred – Image flow from left camera, 60° from central camera on the left side. - /Right_Camera_blurred – Image flow from right camera, 60° from central camera on the right side. - /camera0/camera_info – Intrinsic and distortion information of left camera. - /camera0/projection_matrix – Extrinsic matrix of left camera from lidar. - /camera2/camera_info – Intrinsic and distortion information of central camera. - /camera2/projection_matrix – Extrinsic matrix of central camera from lidar. - /camera5/camera_info – Intrinsic and distortion information of right camera. - /camera5/projection_matrix – Extrinsic matrix of right camera from lidar. - /novatel/oem7/bestpos – Latitude, longitude, and elevation information from NovAtel GNSS-INS system. - /points_raw – VLP-32 lidar point cloud. - /tf – Vehicle base frame. The data are organized into a data description file and one or more ROS .bag files, dependent on original file size. The data description file provides information about the data collection date, location, and detailed mapping, while the .bag file(s) contain the actual data. Please note that for the purpose of securing personally identifiable information, all license plates and faces included in this publication have been intentionally blurred during real-time processing. ![garfield ridge image](garfield-ridge.png)

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI

Raw_data_Batch_I: Lakeshore Drive

Date of collection: May 18, 2023 Location: Lakeshore Drive, Chicago, IL Content: “North to South” “South to North” This data set contains lidar and vision data collected along Lakeshore Drive. The “South to North” folder starts from the intersection of Lakeshore Drive and 31st Street and ends at Hollywood Towers Chicago. The “North to South” folder starts from the intersection of Lakeshore Drive and Sheridan Avenue and ends at the 31st Street intersection. The data contains the following Robot Operating System (ROS) topics: - /Central_Camera_blurred – Image flow from coaxial camera heading, vehicle front. - /Left_Camera_blurred – Image flow from left camera, 60° from central camera on the left side. - /Right_Camera_blurred – Image flow from right camera, 60° from central camera on the right side. - /camera0/camera_info – Intrinsic and distortion information of left camera. - /camera0/projection_matrix – Extrinsic matrix of left camera from lidar. - /camera2/camera_info – Intrinsic and distortion information of central camera. - /camera2/projection_matrix – Extrinsic matrix of central camera from lidar. - /camera5/camera_info – Intrinsic and distortion information of right camera. - /camera5/projection_matrix – Extrinsic matrix of right camera from lidar. - /novatel/oem7/bestpos – Latitude, longitude, and elevation information from NovAtel GNSS-INS system. - /points_raw – VLP-32 lidar point cloud. - /tf – Vehicle base frame. The data are organized into a data description file and one or more ROS .bag files, dependent on original file size. The data description file provides information about the data collection date, location, and detailed mapping, while the .bag file(s) contain the actual data. Please note that for the purpose of securing personally identifiable information, all license plates and faces included in this publication have been intentionally blurred during real-time processing. ![lakeshore drive image](lakeshore-drive.png)

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI

Raw_data_Batch_I: Lisle to Waterfall Glen

Date of collection: May 11, 2023 Location: DuPage County, IL This dataset contains lidar and vision data collected between Lisle, IL, and the Waterfall Glen parking lot. The vehicle started near Cass School District 63, headed east along IL 34. The vehicle then turned south along IL 83 until Interstate 55. Finally, the vehicle turned southwest along I 55 until Exit 273A and headed toward the Waterfall Glen parking lot. The data contains the following Robot Operating System (ROS) topics: - /Central_Camera_blurred – Image flow from coaxial camera heading, vehicle front. - /Left_Camera_blurred – Image flow from left camera, 60° from central camera on the left side. - /Right_Camera_blurred – Image flow from right camera, 60° from central camera on the right side. - /camera0/camera_info – Intrinsic and distortion information of left camera. - /camera0/projection_matrix – Extrinsic matrix of left camera from lidar. - /camera2/camera_info – Intrinsic and distortion information of central camera. - /camera2/projection_matrix – Extrinsic matrix of central camera from lidar. - /camera5/camera_info – Intrinsic and distortion information of right camera. - /camera5/projection_matrix – Extrinsic matrix of right camera from lidar. - /novatel/oem7/bestpos – Latitude, longitude, and elevation information from NovAtel GNSS-INS system. - /points_raw – VLP-32 lidar point cloud. - /tf – Vehicle base frame. The data are organized into a data description file and one or more ROS .bag files, dependent on original file size. The data description file provides information about the data collection date, location, and detailed mapping, while the .bag file(s) contain the actual data. Please note that for the purpose of securing personally identifiable information, all license plates and faces included in this publication have been intentionally blurred during real-time processing. ![lisle waterfall image](lisle-waterfall.png)

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI

Raw_data_Batch_I: Randall Road

Date of collection: June 3, 2022 Location: Randall Road, DuPage County, IL Content: “North to South” “South to North” This data set contains lidar and vision data collected along Randall Road in DuPage County, Illinois. The “South to North” folder starts at 1480 N. Orchard Road, Aurora, IL 60506, headed north along Randall Road until 238 N. Randall Road, St. Charles, IL 60174. The “North to South” folder starts from 238 N. Randall Road, St. Charles, IL 60174, headed south along Randall Road until 1480 N. Orchard Road, Aurora, IL. The data contains the following Robot Operating System (ROS) topics: - /Central_Camera_blurred – Image flow from coaxial camera heading, vehicle front. - /Left_Camera_blurred – Image flow from left camera, 60° from central camera on the left side. - /Right_Camera_blurred – Image flow from right camera, 60° from central camera on the right side. - /camera0/camera_info – Intrinsic and distortion information of left camera. - /camera0/projection_matrix – Extrinsic matrix of left camera from lidar. - /camera2/camera_info – Intrinsic and distortion information of central camera. - /camera2/projection_matrix – Extrinsic matrix of central camera from lidar. - /camera5/camera_info – Intrinsic and distortion information of right camera. - /camera5/projection_matrix – Extrinsic matrix of right camera from lidar. - /novatel/oem7/bestpos – Latitude, longitude, and elevation information from NovAtel GNSS-INS system. - /points_raw – VLP-32 lidar point cloud. - /tf – Vehicle base frame. The data are organized into a data description file and one or more ROS .bag files, dependent on original file size. The data description file provides information about the data collection date, location, and detailed mapping, while the .bag file(s) contain the actual data. Please note that for the purpose of securing personally identifiable information, all license plates and faces included in this publication have been intentionally blurred during real-time processing. ![randall road image](randall-road.png)

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI

Raw_data_Batch_I: State Street

Date of collection: May 18, 2023 Location: State Street, Chicago, IL This data set contains lidar and vision data collected along State Street. The vehicle started from outside of the McCormick Tribune Campus Center at the Illinois Institute of Technology’s Mies Campus and headed north along State Street, until the north end of State Street in the Gold Coast. The data contains the following Robot Operating System (ROS) topics: - /Central_Camera_blurred – Image flow from coaxial camera heading, vehicle front. - /Left_Camera_blurred – Image flow from left camera, 60° from central camera on the left side. - /Right_Camera_blurred – Image flow from right camera, 60° from central camera on the right side. - /camera0/camera_info – Intrinsic and distortion information of left camera. - /camera0/projection_matrix – Extrinsic matrix of left camera from lidar. - /camera2/camera_info – Intrinsic and distortion information of central camera. - /camera2/projection_matrix – Extrinsic matrix of central camera from lidar. - /camera5/camera_info – Intrinsic and distortion information of right camera. - /camera5/projection_matrix – Extrinsic matrix of right camera from lidar. - /novatel/oem7/bestpos – Latitude, longitude, and elevation information from NovAtel GNSS-INS system. - /points_raw – VLP-32 lidar point cloud. - /tf – Vehicle base frame. The data are organized into a data description file and one or more ROS .bag files, dependent on original file size. The data description file provides information about the data collection date, location, and detailed mapping, while the .bag file(s) contain the actual data. Please note that for the purpose of securing personally identifiable information, all license plates and faces included in this publication have been intentionally blurred during real-time processing. ![state street image](state-street.png)

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI

Measurement of the 1-jettiness event shape observable in deep-inelastic electron-proton scattering at HERA

The H1 Collaboration reports the first measurement of the 1-jettiness event shape observable $τ^b_1$ in neutral-current deep-inelastic electron-proton scattering (DIS). The observable $τ^b_1$ is equivalent to a thrust observable defined in the Breit frame. The data sample was collected at the HERA ep collider in the years 2003–2007 with center-of-mass energy of $\sqrt{s}$ = 319 GeV, corresponding to an integrated luminosity of 351.1 pb -1 . Triple differential cross sections are provided as a function of $τ^b_1$, event virtuality $Q^2$, and inelasticity y, in the kinematic region $Q^2$ > 150 GeV 2 . Single differential cross section are provided as a function of $τ^b_1$ in a limited kinematic range. Double differential cross sections are measured, in contrast, integrated over $τ^b_1$ and represent the inclusive neutral-current DIS cross section measured as a function of $Q^2$ and y. The data are compared to a variety of predictions and include classical and modern Monte Carlo event generators, predictions in fixed-order perturbative QCD where calculations up to $\mathcal{O}$($α^3_s$) are available for $τ^b_1$ or inclusive DIS, and resummed predictions at next-to-leading logarithmic accuracy matched to fixed order predictions at $\mathcal{O}$($α^2_s$). These comparisons reveal sensitivity of the 1-jettiness observable to QCD parton shower and resummation effects, as well as the modeling of hadronization and fragmentation. Within their range of validity, the fixed-order predictions provide a good description of the data. Monte Carlo event generators are predictive over the full measured range and hence their underlying models and parameters can be constrained by comparing to the presented data.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS

Charm fragmentation fractions and $\mathbf {c\overline{c}}$ cross section in p–Pb collisions at $\mathbf {\sqrt{s _{NN}}=5.02 TeV}$

The total charm-quark production cross section per unit of rapidity dσ($\mathbf {c\overline{c}}$)/dy, and the fragmentation fractions of charm quarks to different charm-hadron species f(c → h c ), are measured for the first time in p–Pb collisions at $\sqrt{s_{NN}}$ = 5.02 TeV at midrapidity (–0.96 < y < 0.04 in the centre-of-mass frame) using data collected by ALICE at the CERN LHC. The results are obtained based on all the available measurements of prompt production of ground-state charm-hadron species: D 0 , D + , D$^{+}_{s}$, and J/ψ mesons, and Λ$^{+}_{c}$ and $Ξ$$^{0}_{c}$ baryons. The resulting cross section is dσ($\mathbf {c\overline{c}}$)/dy = 219.6 ± 6.3 (stat.) $^{+10.5}_{–11.8}$ (syst.) $^{+8.3}_{–2.9}$ (extr.) ± 5.4 (BR) ± 4.6 (lumi.) ± 19.5 (rapidity shape) + 15.0 (Ω$^{0}_{c}$) mb, which is consistent with a binary scaling of pQCD calculations from pp collisions. The measured fragmentation fractions are compatible with those measured in pp collisions at $\sqrt{s_{NN}}$ = 5.02 and 13 TeV, showing an increase in the relative production rates of charm baryons with respect to charm mesons in pp and p–Pb collisions compared with e + e – and e – p collisions. The p T -integrated nuclear modification factor of charm quarks, R pPb ($\mathbf {c\overline{c}}$) = 0.91 ± 0.04 (stat.) $^{+0.08}_{–0.09}$ (syst.) $^{+0.05}_{–0.03}$ (extr.) ± 0.03 (lumi.), is found to be consistent with unity and with theoretical predictions including nuclear modifications of the parton distribution functions.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS

Serial-femtosecond crystallography reveals how a phytochrome variant couples chromophore and protein structural changes

The photoreaction and commensurate structural changes of a chromophore within biological photoreceptors elicit conformational transitions of the protein promoting the switch between deactivated and activated states. We investigated how this coupling is achieved in a bacterial phytochrome variant, Agp2-PAiRFP2. Contrary to classical protein crystallography, which only allows probing (cryo-trapped) stable states, we have used time-resolved serial femtosecond x-ray crystallography (tr-SFX) and pump-probe techniques with various illumination and delay times with respect to photoexcitation of the parent Pfr state. Thus, structural data for seven time frames were sorted into groups of molecular events along the reaction coordinate. They range from chromophore isomerization to the formation of Meta-F, the intermediate that precedes the functional relevant secondary structure transition of the tongue. Structural data for the early events were used to calculate the photoisomerization pathway to complement the experimental data. Late events allow identifying the molecular switch that is linked to the intramolecular proton transfer as a prerequisite for the following structural transitions.

59 BASIC BIOLOGICAL SCIENCES

Raw Lidar and Camera Data Synchronized with Precipitation and Present Weather Data

As part of the sensor characterization task of the SMART 2.0 project, this dataset includes raw data from three spinning lidars ([Ouster OS2-128](https://ouster.com/products/scanning-lidar/os2-sensor/), [Velodyne Puck (VLP-16)](https://velodynelidar.com/products/puck/), and [Velodyne Ultra Puck (VLP-32)](https://velodynelidar.com/products/ultra-puck/)), one camera ([Mako G-319](https://www.alliedvision.com/en/camera-selector/detail/mako/g-319/)), and one present weather sensor ([Vaisala FD-70](https://www.vaisala.com/en/products/weather-environmental-sensors/forward-scatter-fd70)). All data were synchronized, with the log start time indicated in the file name (HHMMSS). The data can be filtered by date, log time (HHMMSS), sensor, frame ID, and weather classification. These data were gathered statically at the Argonne Testbed for Multiscale Observational Science (ATMOS). Two target stop signs were placed in view of the sensors to contribute a target for comparing sensor data under different conditions. The weather data for each day are stored in netCDF “.nc” files. The lidar data contain the X, Y, Z, intensity, reflectivity, and ring from Ouster OS2-128 rev6, Velodyne VLP-16, and Velodyne VLP-32 lidars. ![raw lidar image](LiDAR_pointcloud_ATMOS.png)

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI

DICOMs, Missiles, and Metadata: The U.S. Nuclear Weapons Program Leverages a Medical Standard

Digital Imagine and Communications in Medicine (DICOM) images, although most commonly used in medical settings, have been widely adopted by the United States Department of Energy (DOE) for capturing images of the internal components in a nuclear weapon. DICOMs, a lesser-known file format that combine nested metadata structures with complex image data-including multiple planes, frames, and high resolution-require the creation of access copies to support usability within the DOE. Used primarily for ensuring the safety, security, and reliability of the U.S. nuclear stockpile, the Los Alamos National Laboratory (LANL)’s DICOM images and corresponding image metadata must be accessible to scientists and researchers via our institutional centralized databases. This paper describes the author's creation of a Python script that converts DICOM images into accessible, archive-friendly TIFF files while preserving key image data and metadata.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION

Addressing Issues with Working Memory in Video Object Segmentation

Contemporary state-of-the-art video object segmentation (VOS) models compare incoming unannotated images to a history of image-mask relations via affinity or cross-attention to predict object masks. We refer to the internal memory state of the initial image-mask pair and past image-masks as a working memory buffer. While the current state of the art models perform very well on clean video data, their reliance on a working memory of previous frames leaves room for error. Affinity-based algorithms include the inductive bias that there is temporal continuity between consecutive frames. To account for inconsistent camera views of the desired object, working memory models need an algorithmic modification that regulates the memory updates and avoid writing irrelevant frames into working memory. A simple algorithmic change is proposed that can be applied to any existing working memory-based VOS model to improve performance on inconsistent views, such as sudden camera cuts, frame interjections, and extreme context changes. The resulting model performances show significant improvement on video data with these frame interjections over the same model without the algorithmic addition. Our contribution is a simple decision function that determines whether working memory should be updated based on the detection of sudden, extreme changes and the assumption that the object is no longer in frame. By implementing algorithmic changes, such as this, we can increase the real-world applicability of current VOS models.

97 MATHEMATICS AND COMPUTING

Alpha-decay width of a near proton-threshold resonance in the 7 Li( α, α ) channel

We investigate the alpha-decay width of a near proton-threshold resonance in 11 B by the excitation functions of 7 Li(α, α) and 7 Li(α, α’) reactions. This resonance is an example of loosely bound atomic nuclei, understood as small open quantum systems, that exhibit properties significantly influenced by their coupling to the continuum. The present experiment focuses on the formation of a specific state in 11 B at excitation energy in the region near 11.4 MeV by bombarding a 7 Li target with a 4 He beam at energies ranging from 3.92 to 4.56 MeV in the laboratory frame. The study provides data complementary to previous observations of the proton emission in the β-decay of the neutron-rich halo nucleus 11 Be. An R-matrix fit to the data provides resonance energy and partial widths consistent with J π =1/2 + for a narrow near-threshold state in 11 B at 11.400(20) MeV, that is 171(20) keV above the proton emission threshold, with partial α-decay width Γ α =2.5$^{+3.5}_{–1.5}$ keV in elastic scattering and Γ α' =15.8$^{+1.9}_{–0.4}$ keV in inelastic scattering. These findings contribute to the understanding of the structure of this open quantum system resonance.

11B

The 4D Camera: An 87 kHz Direct Electron Detector for Scanning/Transmission Electron Microscopy

We describe the development, operation, and application of the 4D Camera—a 576 by 576 pixel active pixel sensor for scanning/transmission electron microscopy which operates at 87,000 Hz. The detector generates data at ~480 Gbit/s which is captured by dedicated receiver computers with a parallelized software infrastructure that has been implemented to process the resulting 10–700 Gigabyte-sized raw datasets. The back illuminated detector provides the ability to detect single electron events at accelerating voltages from 30 to 300 kV. Through electron counting, the resulting sparse data sets are reduced in size by 10--300× compared to the raw data, and open-source sparsity-based processing algorithms offer rapid data analysis. The high frame rate allows for large and complex scanning diffraction experiments to be accomplished with typical scanning transmission electron microscopy scanning parameters.

47 OTHER INSTRUMENTATION

Geospatial Diffusion for Land Cover Imperviousness Change Forecasting

Land-use and land-cover (LULC) has a significant effect on several Earth system processes. For example, impervious surfaces reduce infiltration and speed water flow, impacting regional hydrology and flood risk. While Earth System models have improved forecasting hydrologic and atmospheric processes at higher resolutions, the ability to forecast LULC change has lagged behind. In this paper, we propose a new paradigm exploiting Generative AI (GenAI) for land cover change forecasting by framing it as a data synthesis problem conditioned on historical and auxiliary data-sources. To demonstrate the feasibility of our methodology, we perform experiments where a diffusion model is trained for decadal forecasting of imperviousness change across the entire United States. We find that our model yields MAE lower than a no-change baseline for resolutions ≥ 0.7 X 0.7km2 on average, demonstrating its ability to capture and project accurate spatiotemporal patterns. Finally, we discuss future research to incorporate Earth's physical properties and enabling scenario simulations via driver variables.

Varshney, Debvrat [ORNL] (ORCID:0000000188981736)

Spatiotemporal Automatic Calibration of Infrastructure Lidar, Radar, and Camera with a Global Navigation Satellite System

Robust and accurate perception is important for modern intelligent transportation systems (ITS), which use sensors of various modalities for data fusion to create a digital twin of an intersection. Sensor calibration is an important process that creates a unified coordinate frame for the sensor output data so that it can be used for data fusion. Classical approaches for sensor calibration are time-consuming, require an overlapping field of view for feature matching, and are not feasible for ITS application as they cause disruptions in the flow of traffic. In this paper, we present a spatiotemporal automatic calibration approach to calibrate multiple infrastructure lidar, radar, and cameras installed at a traffic intersection. The approach uses global navigation satellite system (GNSS) positioning information shared by connected vehicles, and when the vehicle is detected by the sensor, we match the sensor detections with the GNSS coordinates. The proposed algorithm is evaluated with a real-world dataset utilizing detections from two radars, cameras, and lidars with a test vehicle instrumented with a post-processing kinematic (PPK)-corrected GNSS driving past the sensors installed at a four-way traffic intersection. The experimental results show that the proposed automatic calibration approach can achieve the transformation with a root mean squared error of less than 0.5 for radar and lidar and less than 2 for camera detections. The ability to rapidly calibrate sensors not only benefits initial installations, but can also be used for system health monitoring, while utilizing available connected vehicle data to test the real-time sensor fidelity and operational status.

ADVANCED PROPULSION SYSTEMS,ENERGY CONSERVATION, C

Spatiotemporal Automatic Calibration of Infrastructure Lidar, Radar, and Camera with a Global Navigation Satellite System: Preprint

Robust and accurate perception is important for modern intelligent transportation systems (ITS), which use sensors of various modalities for data fusion to create a digital twin of an intersection. Sensor calibration is an important process that creates a unified coordinate frame for the sensor output data so that it can be used for data fusion. Classical approaches for sensor calibration are time-consuming, require an overlapping field of view for feature matching, and are not feasible for ITS application as they cause disruptions in the flow of traffic. In this paper, we present a spatiotemporal automatic calibration approach to calibrate multiple infrastructure lidar, radar, and cameras installed at a traffic intersection. The approach uses global navigation satellite system (GNSS) positioning information shared by connected vehicles, and when the vehicle is detected by the sensor, we match the sensor detections with the GNSS coordinates. The proposed algorithm is evaluated with a real-world dataset utilizing detections from two radars, cameras, and lidars with a test vehicle instrumented with a post-processing kinematic (PPK)-corrected GNSS driving past the sensors installed at a four-way traffic intersection. The experimental results show that the proposed automatic calibration approach can achieve the transformation with a root mean squared error of less than 0.5 for radar and lidar and less than 2 for camera detections. The ability to rapidly calibrate sensors not only benefits initial installations, but can also be used for system health monitoring, while utilizing available connected vehicle data to test the real-time sensor fidelity and operational status.

ADVANCED PROPULSION SYSTEMS

Modeling performance of data collection systems for high-energy physics

Exponential increases in scientific experimental data are outpacing silicon technology progress, necessitating heterogeneous computing systems—particularly those utilizing machine learning (ML)—to meet future scientific computing demands. The growing importance and complexity of heterogeneous computing systems require systematic modeling to understand and predict the effective roles for ML. We present a model that addresses this need by framing the key aspects of data collection pipelines and constraints and combining them with the important vectors of technology that shape alternatives, computing metrics that allow complex alternatives to be compared. For instance, a data collection pipeline may be characterized by parameters such as sensor sampling rates and the overall relevancy of retrieved samples. Alternatives to this pipeline are enabled by development vectors including ML, parallelization, advancing CMOS, and neuromorphic computing. By calculating metrics for each alternative such as overall F1 score, power, hardware cost, and energy expended per relevant sample, our model allows alternative data collection systems to be rigorously compared. We apply this model to the Compact Muon Solenoid experiment and its planned high luminosity-large hadron collider upgrade, evaluating novel technologies for the data acquisition system (DAQ), including ML-based filtering and parallelized software. The results demonstrate that improvements to early DAQ stages significantly reduce resources required later, with a power reduction of 60% and increased relevant data retrieval per unit power (from 0.065 to 0.31 samples/kJ). However, we predict that further advances will be required in order to meet overall power and cost constraints for the DAQ.

Olin-Ammentorp, Wilkie (ORCID:0000000224729862)

Spectral kernel machines with electrically tunable photodetectors

Spectral machine vision collects spectral and spatial information as three-dimensional hypercubes and digitally processes them, which causes a data bottleneck, limiting power efficiency, frame rate, and spectral-spatial resolution. This work introduces spectral kernel machines (SKMs) to overcome these bottlenecks. SKM directly compresses spectral analysis through the output photocurrent and learns from example objects to identify and classify new samples in a "sniff-and-seek" mode. We experimentally demonstrated SKMs with electrically tunable bipolar black phosphorus-molybdenum disulfide (bP-MoS2) photodiodes in the near- and mid-infrared band and silicon photoconductors in the visible band, performing versatile intelligent tasks from chemometrics to semiconductor metrology. This architecture consumed substantially less power and was more than an order of magnitude faster than existing solutions for hyperspectral image analysis, defining an intelligent imaging and sensing paradigm with intriguing possibilities.

Zhang, Dehui