LargeScaleMeanShift segmentation of hyperspectral data

OTB Usage
1

Hello, I’m a new user to OrfeoToolBox AND hyperspectral imagery, so I’m hoping to learn quite a bit from this community :slight_smile:

I’m attempting to use LargeScaleMeanShift to segment hyperspectral imagery collected by drone, provided in ENVI format. I have attached a small clip of the data with header file. I got the idea from this paper.

My issue is that I am unable to get LargeScaleMeanShift to segment the data in its original format. If I include the total number of spectral bands in the data (328 bands), LargeScaleMeanShift returns the segmentation vector output as a single polygon positioned in the upper left corner of the region where the data were collected (see uploaded image). The dimensions of the polygon equal my input for parameters “size of tiles in pixel (X or Y axis)”. Label maps produced by this process denote a single label value for the entire region. I thought originally this could be a memory issue, as the original ENVI file is 5 GB, but even a much smaller clip of the data (header file attached, data file is ~35MB) does not produce proper segmentation.

However, I am able to get segmentation if I reduce the number of bands to three, just to cover RGB, and save the data as a non-georeferenced image (a PNG, JIF, or TIF for example) rather than as a georeferenced ENVI or GeoTIFF file. Unfortunately, I hope to a) keep the data as a georeferenced object and b) use all the separate spectral bands to inform segmentation, rather than just three. Segmentation also does not occur if I reduce the number of bands but save the data in georeferenced ENVI or GeoTIFF format.

I have tried several times to perform segmentation by using alternative raster formats and (arbitrarily!) testing the input parameters for LargeScaleMeanShift (spatial radius, range radius, minimum segment size, size of tiles in pixel) to no avail. The only thing that seems to work for me is to give it a non-georeferenced rgb image. My understanding of this tool is limited so I assume that I am setting things up incorrectly, though I am following instructions in the LargeScaleMeanShift online doc.
After reviewing the online doc, I decided to run the different components of LargeScaleMeanShift independently (MeanShiftSmoothing, LSMSSegmentation, LSMSSmallRegionsMerging and LSMSVectorization). MeanShiftSmoothing works fine, but LSMSSegmentation does not produce expected results.

I am using OTB version 7.1 on a Windows 10 machine, and loading the segmentation tool through the Monteverdi application launcher GUI.

Thanks for any advice you can pass along.

raw_0_clip7-6.hdr (3.6 KB)

output
output1263×836 416 KB

2

Hello Sam and welcome to Orfeo ToolBox !

First, I’m not sure if processing a segmentation over a 300+ bands image makes sense : as you’ve noticed, you have to fix a range radius (dynamic range) and it may be difficult to find a common range radius for all the bands. It works fine on RGB images because usually we know the range of the values.

Another point I don’t understand is that the segmentation final output is a vector file, containing polygons that overlay pixels in a neighboorhood (depends on spatial radius) and close values (range radius). You can use whatever kind of vector file handled by GDAL (like shp, sqlite, gkpg, etc.).
If you use the different steps separately (which is a good option to learn more and fix all parameters), the intermediate files will be raster files and OTB can write any kind of georeferenced raster file (here again, it’s based on GDAL).

You can learn more about Large Scale Mean Shift Segmentation by reading the recipes in the CookBook : https://www.orfeo-toolbox.org/CookBook/recipes/improc.html#large-scale-mean-shift-lsms-segmentation

Depending on what you try to do (what kind of information you want to extract from your image), you shall have a look at the classification framework, maybe with a non-supervised approach (like K-Means classification). But I think that you shall first select relevant bands in your image.

Hope that this can help you, and do not hesitate to share other questions or give us some more information so we can reproduce what you are trying to do.

Best regards

Yannick

2 Likes
3

Hi Yannick, thank you for your help. I was able to get this to work with any number of bands but segmentation seems to work best with fewer bands. I did find issues with running segmentation based on ENVI format and had to convert to geotiff, though maybe that’s just due to an issue with my specific file.

Cheers
Sam