Sky detection using Slic superpixels

Sky detection using Slic superpixels

Juraj Kostolansky

This project tries to solve the problem of sky detection using the Slic superpixel segmentation algorithm.

Analysis

The first idea was to use Slic superpixel algorithm to segment an input image and merge pairs of adjecent superpixels based on their similarity. We created a simple tool to manually evaluate the hypothesis that a sky can be separated from a photo with one threshold. In this prototype, we compute the similarity between superpixels as an Euclidean distance between their mean colors in the RGB color space. For the most of images from our dataset we found a threshold which can be used for sky segmentation process.

Next, we analyzed colors of images from our dataset. For each image we saved superpixel colors for sky and the rest of the image in three color spaces (RGB, HSV and Lab) and we plotted them. The resulting graphs are shown below (the first row of graphs represents sky colors, the second row represents colors of the rest of an image). As we can see, the biggest difference is in HSV an Lab color spaces. Based on this evaluation, we choosed Lab as a base working color space to compare superpixels.

RGB

kostolansky_rgb

HSV

kostolansky_hsv

Lab
kostolansky_lab

Final algorithm

  1. Generating superpixels using Slic algorithm
  2. Replacing superpixels with their mean color values
  3. Setting threshold:
    T = [ d1 + (d2 - d1) * 0.3 ] * 1.15
    

    where:

    • d1 – average distance between superpixels in the top 10% of an image
    • d2 – average distance between superpixels in an image without ⅓ smallest distances

    The values 0,3 and 1,15 was choosed for best (universal) results for our dataset.

  4. Merging adjecent superpixels
  5. Choosing sky – superpixel with the largest number of pixels in the first row
  6. Draw sky border (red)

Sample

kostolansky_a

kostolansky_b

kostolansky_c

kostolansky_d