Per-pixel classification of clouds from whole sky HDR images

Accurately identifying cloud types in images has multiple uses from meteorological science to computer graphics, especially as clouds are a major factor influencing atmospheric radiative transport. Understanding which cloud types are present in an image is typically performed on a coarse scale, where cloud types are identified per image, but do not permit a finer, per-pixel granularity of labelling cloud types. This paper presents a novel approach which solves this problem via a per-pixel classification method for identifying cloud types based on High Dynamic Range imagery of skies. The proposed method requires minimal labelling of the training data, and utilizes a hierarchical patch-based feature extraction technique which describes the statistical and structural features about regions of the image. This enables the extraction of representative feature vectors which are used for subsequent labelling. This approach is the first to produce a per-pixel classification of cloud types from a single image, with an accuracy of 84%. Additionally, when applied to whole sky cloud classification, our results produce a 98.3% accuracy, which is competitive with the state-of-the-art.