The results from training a U-net on dataset v0.0.1 were better than feared. I thought there would be too much noise and bad iterations from the dataset, as mentioned in the previous post. Despite the discussed issues, the network handled the dataset surprisingly well:
Training metrics:
Predictions:
As can be seen on set number two on the figures above, another issue with the dataset occurred. When extracting sets, some of the raster maps were extracted out of bounds from the aerial imagery base. This resulted in some black parts of the maps. Another observation is that several sets contain images with the background class only. An important part of the network is distinguishing between buildings and water and what is not. However, these sets might only prolong the training time, as many pixels belonging to the background class exists in the sets containing the target classes. This also translates to sets containing only water. I believe that there are enough sets with both target classes than for these sets to be included.
Bumping the dataset to v0.1.0
Regarding what i just mentioned, the new dataset has no sets with images that contain black pixels(0,0,0) as these are out of bounds sets and might induce noise. Sets containing only background class or only water class have also been removed. Lastly the masks have synthetically changed their interpolation to nearest neighbour, and converted to grayscale images. This reduces their size, and removes any ambiguous pixels between object borders that were not part of any classes. I.e buildings etc in masks have hard edges, and not smooth transitions. Such transitions may affect training negatively, and cause problems.
The results from said improvements were a reduction of the dataset size from 25,1Gb to 17,4Gb, and from 67219 to 43802 samples. There are still improvements to be made on the variance of the dataset and the annotations’ accuracy. Said improvements are the goal for the next version.
Statistics:
- Mean: [114.84165988 115.00874479 104.93408015]
- Standard deviation: [47.1493354 42.40584163 38.70938704]
- Building pixels: 2787227245
- Water pixels: 744719009
- Background pixels: 7950485234
- Building percentage: 24 %
- Water pixels: 6 %
- Background pixels: 69 %