A posteriori clear-sky identification methods in solar irradiance time series: Review and preliminary validation using sky imagers
Not Available
Gueymard, Christian A.↗
SEARCH · Search NASA
Results for “sky imaging”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Automated construction of clear-sky dictionary from all-sky imager data
All-sky imagers (ASIs) have significant promise as scalable sensors for short-term solar irradiance forecasting. Many of the current computational techniques that use ASIs for this purpose rely on collections of clear-sky images indexed by time of day, solar angle, or both, called clear-sky dictionaries (CSDs). These CSDs act as baselines against which images can be compared to locate and classify clouds within the image frame. CSDs are often compiled by hand, where individuals visually inspect collections of images one at a time to find clear-sky images. This process is not scalable, and it is prone to error. This paper proposes an automated, nonparametric alternative that uses the principles of digital image processing to find clear-sky images within a set of images taken over several days. We use ground-truth measurements of the clearness index to assess the performance of our method, and we show that the images it selects accurately correspond to clear-sky images. We also compare our proposal, which is nonparametric, with a state-of-the-art parametric method. The numerical results indicate that the performance of the method proposed here is superior.
14 SOLAR ENERGY↗
Infrared Sky Imager (IRSI) Instrument Handbook
The Infrared Sky Imager (IRSI) deployed at the Atmospheric Radiation Measurement (ARM) Climate Research Facility is a Solmirus Corp. All Sky Infrared Visible Analyzer. The IRSI is an automatic, continuously operating, digital imaging and software system designed to capture hemispheric sky images and provide time series retrievals of fractional sky cover during both the day and night. The instrument provides diurnal, radiometrically calibrated sky imagery in the mid-infrared atmospheric window and imagery in the visible wavelengths for cloud retrievals during daylight hours. The software automatically identifies cloudy and clear regions at user-defined intervals and calculates fractional sky cover, providing a real-time display of sky conditions.
54 ENVIRONMENTAL SCIENCES↗
University of California, San Diego (UCSD) Sky Imager Cloud Position Study Field Campaign Report
During the University of California, San Diego (UCSD) Sky Imager Cloud Position Study, two University of California, San Diego Sky Imagers (USI) (Figure 1) were deployed the U.S. Department of Energy(DOE)’s Atmospheric Radiation Measurement (ARM) Climate Research Facility Southern Great Plains SGP) research facility. The UCSD Sky Imagers were placed 1.7 km apart to allow for stereographic determination of the cloud height for clouds over approximately 1.5 km. Images with a 180-degree field of view were captured from both systems during daylight hours every 30 seconds beginning on March 11, 2013 and ending on November 4, 2013. The spatial resolution of the images was 1,748 × 1,748, and the intensity resolution was 16 bits using a high-dynamic-range capture process. The cameras use a fisheye lens, so the images are distorted following an equisolid angle projection.
58 GEOSCIENCES↗
A Novel Segmentation Algorithm for the ARM User Facility All-Sky Imagers Using Machine Learning Applications
Cloud cover plays a pivotal role in modulating the Earth's energy budget through the reflection of incoming solar radiation and the trapping of outgoing longwave radiation. Ground-based all-sky imagers offer an objective assessment of cloud cover that can be used to estimate solar irradiance, classify cloud types, track cloud movement, and serve as a benchmark 10 for the evaluation of satellite and reanalysis data products. The Atmospheric Radiation Measurement (ARM) user facility has utilized all-sky imagers for more than 25 years to monitor cloud cover and augment its comprehensive suite of atmospheric measurements. Following the retirement of its Total Sky Imager (TSI), ARM recently deployed the TSI’s successor, the All Sky Imager (ASI-16 camera systems). To provide a smooth transition and continuity to the vast amount of knowledge gathered by the TSI over the years, while addressing typical deployment issues, we developed a novel pixel segmentation algorithm, 15 the ASI Sky Cover (ASISKYCOVER). ASISKYCOVER builds on the different strengths and properties of the TSI processing algorithm while integrating machine learning techniques, ensuring data validity and accuracy across diverse atmospheric conditions. It enhances cloud cover characterization with new features such as artifact detection and uncertainty quantification. ASISKYCOVER also includes cloud cover estimates for near-zenith (narrow field-of-view) and reduces susceptibility to false detections. This study introduces ASISKYCOVER, details its algorithm framework, and demonstrates its capabilities using a 20 year-long dataset from the ARM Southern Great Plains site. Comparisons with co-located TSI data and other ARM measurements, such as zenith-pointing radars and lidars, are presented, underscoring the ASISKYCOVER’s potential to improve cloud cover analyses and data evaluation efforts, as well as to be integrated into higher-level data products that synergize instrument suites to generate new and insightful information
Silber, Israel↗
Automatic Detection and Classification of Aurora in THEMIS All‐Sky Images
We report a novel machine-learning algorithm for automatically detecting and classifying aurora in all–sky images (ASI) that is largely trained without requiring ground–truth labels. By including a small number of labeled images, we are able to automatically label all of the approximately 700 million images in the Time History of Events and Macroscale Interactions during Substorms (THEMIS) ASI data set from 2008 to 2022. We use a two–stage approach. In the first stage, we adapt the Simple framework for Contrastive Learning of Representations (SimCLR) algorithm to learn latent representations of THEMIS all–sky images. We then finetune a classifier network on the latent representations our model learns of the manually labeled Oslo aurora THEMIS (OATH) data set. We demonstrate that this two–stage approach achieves excellent classification results on data for which there is no current ML classification benchmark. The outcome of this work will facilitate efficient information retrieval for researchers interested in specific categories of aurora and will enable large scale statistical studies and machine learning analyses of THEMIS all–sky images that have not previously been possible. To demonstrate possible ways to utilize this database, we performed a statistical analysis of the occurrence rates of auroral labels with respect to solar wind parameters, interplanetary magnetic field vector, and geomagnetic indices. We further investigate the occurrence rates of auroral phenomena in the annotated data set and their geoeffectiveness by utilizing the co–located THEMIS ground magnetometer data set.
Jeremiah W Johnson↗
Convolutional neural networks for intra-hour solar forecasting based on sky image sequences
Accurate and timely solar forecasts play an increasingly critical role in power systems. Compared to longer forecasting timescales, very short-term solar forecasting has lagged behind in both research and practice. In this paper, we propose deep convolutional neural networks (CNNs) to provide operational intra-hour (10-minute-ahead to 60-minute-ahead) solar forecasts. We develop two CNN structures inspired by a widely-used CNN architecture. The CNNs are tailored to our solar forecasting regression tasks and rely solely on sky image sequences. Case studies based on six years of data (over 150,000 data points) demonstrate that the best CNN model has forecast skill scores of 20%-39% over the naive persistence of cloudiness benchmark, even at these very short timescales. The CNNs also have consistently superior performance when compared to shallow machine learning models with meteorological predictors, where the improvement averages around 7%. The sensitivity analyses show that the sky image length, resolution, and weather conditions have impacts on the deep learning model accuracy. In our intra-hour problem with specific setups, two sky images with a 10-minute 128 x 128 resolution yield the most accurate forecasts. Current limitations, future work, and deployment challenges and solutions are also discussed.
14 SOLAR ENERGY↗
Shallow cumuli cover and its uncertainties from ground-based lidar–radar data and sky images
Cloud cover estimates of single-layer shallow cumuli obtained from narrow field-of-view (FOV) lidar–radar and wide-FOV total sky imager (TSI) data are compared over an extended period (2000–2017 summers) at the established United States Atmospheric Radiation Measurement mid-continental Southern Great Plains site. We quantify the impacts of two factors on hourly and sub-hourly cloud cover estimates: (1) instrument-dependent cloud detection and data merging criteria and (2) FOV configuration. Enhanced observations at this site combine the advantages of the ceilometer, micropulse lidar (MPL) and cloud radar in merged data products. Data collected by these three instruments are used to calculate narrow-FOV cloud fraction (CF) as a temporal fraction of cloudy returns within a given period. Sky images provided by TSI are used to calculate the wide-FOV fractional sky cover (FSC) as a fraction of cloudy pixels within a given image. To assess the impact of the first factor on CF obtained from the merged data products, we consider two additional subperiods (2000–2010 and 2011–2017 summers) that mark significant instrumentation and algorithmic advances in the cloud detection and data merging. We demonstrate that CF obtained from ceilometer data alone and FSC obtained from sky images provide the most similar and consistent cloud cover estimates; hourly bias and root-mean-square difference (RMSD) are within 0.04 and 0.12, respectively. However, CF from merged MPL–ceilometer data provides the largest estimates of the multiyear mean cloud cover, about 0.12 (35 %) and 0.08 (24 %) greater than FSC for the first and second subperiods, respectively. CF from merged ceilometer–MPL–radar data has the strongest subperiod dependence with a bias of 0.08 (24 %) compared to FSC for the first subperiod and shows no bias for the second subperiod. The strong period dependence of CF obtained from the combined ceilometer–MPL–radar data is likely results from a change in what sensors are relied on to detect clouds below 3 km. After 2011, the MPL stopped being used for cloud top height detection below 3 km, leaving the radar as the only sensor used in cloud top height retrievals. To quantify the FOV impact, a narrow-FOV FSC is derived from the TSI images. We demonstrate that FOV configuration does not modify the bias but impacts the RMSD (0.1 hourly, 0.15 sub-hourly). In particular, the FOV impact is significant for sub-hourly observations, where 41 % of narrow- and wide-FOV FSC differ by more than 0.1. A new “quick-look” tool is introduced to visualize impacts of these two factors through integration of CF and FSC data with novel TSI-based images of the spatial variability in cloud cover. The influence of cloud field organization, such cloud streets parallel to the wind direction, on narrow- and wide-FOV cloud cover estimates can be visually assessed.
54 ENVIRONMENTAL SCIENCES↗
Custom Sky-Image Mosaics from NASA's Information Power Grid
yourSkyG is the second generation of the software described in yourSky: Custom Sky-Image Mosaics via the Internet (NPO-30556), NASA Tech Briefs, Vol. 27, No. 6 (June 2003), page 45. Like its predecessor, yourSkyG supplies custom astronomical image mosaics of sky regions specified by requesters using client computers connected to the Internet. Whereas yourSky constructs mosaics on a local multiprocessor system, yourSkyG performs the computations on NASA s Information Power Grid (IPG), which is capable of performing much larger mosaicking tasks. (The IPG is high-performance computation and data grid that integrates geographically distributed 18 NASA Tech Briefs, September 2005 computers, databases, and instruments.) A user of yourSkyG can specify parameters describing a mosaic to be constructed. yourSkyG then constructs the mosaic on the IPG and makes it available for downloading by the user. The complexities of determining which input images are required to construct a mosaic, retrieving the required input images from remote sky-survey archives, uploading the images to the computers on the IPG, performing the computations remotely on the Grid, and downloading the resulting mosaic from the Grid are all transparent to the user
Jacob, Joseph↗
Resampling and data augmentation for short-term PV output prediction based on an imbalanced sky images dataset using convolutional neural networks
Integrating photovoltaics (PV) into electricity grids is challenged by potentially large fluctuations in power generation. In recent years, sky image-based PV output prediction using convolutional neural networks (CNNs) has emerged as a promising approach to forecasting fluctuations. A key challenge is imbalanced sky image datasets: because of the geography of solar PV system installations, sky image datasets are often rich in sunny condition data but deficient in cloudy condition data. This imbalance contrasts with the fact that model errors are dominated by cloudy condition performance. In this study, we attempt to remedy this by exploring the enrichment and augmentation of an imbalanced sky images dataset for two PV output prediction tasks: nowcasting (predicting concurrent PV output) and forecasting (predicting 15-minute-ahead future PV output). We empirically examine the efficacy of using different resampling and data augmentation approaches to create a rebalanced dataset for model development. A three-stage greedy search is used to determine the optimal resampling approach, data augmentation techniques and over-sampling rate. The results show that for the nowcast problem, resampling and data augmentation can effectively enhance the model performance, reducing overall root mean squared error (RMSE) by an average of 4%, or a 15 std. (standard deviation) of improvement compared to the variability of the baseline model. In contrast, the treatment RMSE for the forecast problem nearly always overlaps the baseline performance at the ± 2 std. level. The optimal resampling approach expands on the original dataset by over-sampling the minority cloudy data, with the best results from large over-sampling rate (e.g., 4 ~ 6 times over-sampling of cloudy images).
14 SOLAR ENERGY↗
Coupling sky images with radiative transfer models: a new method to estimate cloud optical depth
A method for retrieving cloud optical depth ( τ c ) using a UCSD developed ground-based sky imager (USI) is presented. The radiance red–blue ratio (RRBR) method is motivated from the analysis of simulated images of various τ c produced by a radiative transfer model (RTM). From these images the basic parameters affecting the radiance and red–blue ratio (RBR) of a pixel are identified as the solar zenith angle ( θ 0 ), τ c , solar pixel angle/scattering angle ( θ s ), and pixel zenith angle/view angle ( θ z ). The effects of these parameters are described and the functions for radiance, I λ τ c , θ 0 , θ s , θ z , and RBR τ c , θ 0 , θ s , θ z are retrieved from the RTM results. RBR, which is commonly used for cloud detection in sky images, provides non-unique solutions for τ c , where RBR increases with τ c up to about τ c = 1 (depending on other parameters) and then decreases. Therefore, the RRBR algorithm uses the measured I λ meas θ s , θ z , in addition to RBR meas θ s , θ z , to obtain a unique solution for τ c . The RRBR method is applied to images of liquid water clouds taken by a USI at the Oklahoma Atmospheric Radiation Measurement (ARM) program site over the course of 220 days and compared against measurements from a microwave radiometer (MWR) and output from the Min et al. (2003) method for overcast skies. τ c values ranged from 0 to 80 with values over 80, being capped and registered as 80. A τ c RMSE of 2.5 between the Min et al. (2003) method and the USI are observed. The MWR and USI have an RMSE of 2.2, which is well within the uncertainty of the MWR. In conclusion, the procedure developed here provides a foundation to test and develop other cloud detection algorithms.
54 ENVIRONMENTAL SCIENCES↗
Short-term solar radiation forecast using total sky imager via transfer learning
Ground-based sky cameras, which capture hemispherical images, have been extensively used for localized monitoring of clouds. This paper proposes a short-term forecasting approach based on transfer learning using Total Sky-Imager (TSI) images of the Southern Great Plains (SGP) site obtained from the Atmospheric Radiation Measurement (ARM) dataset. An accurate estimation of solar irradiance using TSI is key for short-term solar energy generation forecasting and optimal energy consumption planning. We make use of deep neural network architectures such as AlexNet and ResNet-101 to extract the underlying deep convolution features from TSI images and then train using an ensemble learning approach to model and forecast solar radiation. We demonstrate the performance of the proposed approach by showcasing the best and worst cases. Thus, the transfer learning approach significantly reduces the time and resources required for modeling solar radiation. We outperform with reference to another state-of-art technique for solar modeling using TSI images at different forecast lead times.
54 ENVIRONMENTAL SCIENCES↗
Temporal-Spatial Structure of Magnetic Merging at the Magnetopause Inferred from 557.7-nm All-Sky Images
We demonstrate that high-resolution 557.7-nm all-sky images are useful tools for investigating the spatial and temporal evolution of merging on the dayside magnetopause. Analysis of ground and satellite measurements leads us to conclude that high-latitude merging events can occur at multiple sites simultaneously and vary asynchronously on time scales of 30 s to 3 min. Variations of 557.7 nm emissions were observed at a 10 s cadence at Ny-Alesund on 19 December 2001, while significant changes in the IMF clock angle were reaching the magnetopause. The optical patterns are consistent with a scenario in which merging occurs around the rim of the high-latitude cusp at positions dictated by the IMF clock angle. Electrons energized at merging sites represent plausible sources for 557.7 nm emissions in the cusp. Polar observations at the magnetopause have directly linked enhanced fluxes of > or = 0.5 keV electrons with merging. Spectra of electrons responsible for some of the emissions, measured during a DMSP F15 overflight, exhibit "inverted-V" features, indicating further acceleration above the ionosphere. SuperDARN spectral width boundaries, characteristic of open-closed field line transitions, are located at the equatorward edge of the 557.7nm emissions. Optical data suggest that with IMF B(sub Y) > 0, the Northern Hemisphere cusp divides into three source regions. When the IMF clock angle was approx. 150 deg structured 557.7-nm emissions came from east of the 13:00 MLT meridian. At larger clock angles the emissions appeared between 12:00 and 13:00 MLT. No significant 557.7-nm emissions were detected in the prenoon MLT sector. MHD simulations corroborate our scenario, showing that with the observed large dipole-tilt and IMF clock angles, merging sites develop near the front and eastern portions of the high-altitude cusp rim in the Northern Hemisphere and near the western part of the cusp rim in the Southern Hemisphere.
Maynard, N. C.↗
yourSky: Custom Sky-Image Mosaics via the Internet
yourSky (http://yourSky.jpl.nasa.gov) is a computer program that supplies custom astronomical image mosaics of sky regions specified by requesters using client computers connected to the Internet. [yourSky is an upgraded version of the software reported in Software for Generating Mosaics of Astronomical Images (NPO-21121), NASA Tech Briefs, Vol. 25, No. 4 (April 2001), page 16a.] A requester no longer has to engage in the tedious process of determining what subset of images is needed, nor even to know how the images are indexed in image archives. Instead, in response to a requester s specification of the size and location of the sky area, (and optionally of the desired set and type of data, resolution, coordinate system, projection, and image format), yourSky automatically retrieves the component image data from archives totaling tens of terabytes stored on computer tape and disk drives at multiple sites and assembles the component images into a mosaic image by use of a high-performance parallel code. yourSky runs on the server computer where the mosaics are assembled. Because yourSky includes a Web-interface component, no special client software is needed: ordinary Web browser software is sufficient.
Jacob, Joseph↗
Simultaneous observation of auroral substorm onset in Polar satellite global images and ground-based all-sky images
Explore the source record for details and available documents.
Akimasa Ieda↗
A New Display Format Relating Azimuth-Scanning Radar Data and All-Sky Images in 3-D
Here we correlate features in a sequence of all-sky images of 630 nm airglow with the three-dimensional (3-D) structure of electron densities in the F region above Arecibo. Pairs of 180 azimuth scans (using the Gregorian and line feeds) of the two-beam incoherent scatter radar (ISR) have been plotted in cone pictorials of the line-of-sight electron densities. The plots include projections of the 630 nm airglow onto the ground using the same spatial scaling as for the ISR data. Selected sequential images from the night of 16-17 June 2004 correlate ionospheric plasma features with scales comparable to the ISR density-cone diameter. The entire set of over 100 images spanning about eight hours is available as a movie. The correlation between the airglow and the electron densities is not unexpected, but the new display format shows the 3-D structures better than separate 2-D plots in latitude and longitude for the airglow and in height and time for the electron densities. Furthermore, the animations help separate the bands of airglow from obscuring clouds and the star field.
Swartz, Wesley E.↗
3D cloud detection and tracking system for solar forecast using multiple sky imagers
We propose a system for forecasting short-term solar irradiance based on multiple total sky imagers (TSIs). The system utilizes a novel method of identifying and tracking clouds in three-dimensional space and an innovative pipeline for forecasting surface solar irradiance based on the image features of clouds. First, we develop a supervised classifier to detect clouds at the pixel level and output cloud mask. In the next step, we design intelligent algorithms to estimate the block-wise base height and motion of each cloud layer based on images from multiple TSIs. Thus, this information is then applied to stitch images together into larger views, which are then used for solar forecasting. We examine the system’s ability to track clouds under various cloud conditions and investigate different irradiance forecast models at various sites. We confirm that this system can 1) robustly detect clouds and track layers, and 2) extract the significant global and local features for obtaining stable irradiance forecasts with short forecast horizons from the obtained images. Finally, we vet our forecasting system at the 32-megawatt Long Island Solar Farm (LISF). Compared with the persistent model, our system achieves at least a 26% improvement for all irradiance forecasts between one and fifteen minutes.
97 MATHEMATICS AND COMPUTING↗
Systems and methods for single-axis tracking via sky imaging and machine leanring comprising a neural network to determine an angular position of a photovoltaic power system
A system and method is disclosed for solar tracking and controlling an angular position of a photovoltaic power system. The solar tracking system includes an imaging device for capturing images of the sky; a solar position data generating module; and a control system comprising a neural network. The neural network has multiple convolutional layers to generate a first output associated with the images, and a solar position data module. A first dense layer module receives the solar position data and generates a second output. A second dense layer module receives the first output and the second output and generates a concatenated data sequence. A processor is programmed to generate a multi-planar irradiance signal (MPIS) in response to the concatenated data sequence, and determine an angular position of the PV power system and adjust the angular position in response to an angle of maximum irradiance.
Stein, Joshua↗