Orfeo Toolbox TrainImageClassifier stats .xml Error. Trainer not working

OTB Usage
1

Orfeo Toolbox Stats failure

I am trying to train some data with the latest version of Orfeo toolbox (8.0.1) and QGIS version 3.22.5-Białowieża. After computing the image stats as a .xml format file without any special characters in the file name and then using the image stats output in the image trainer, I keep receiving this error. I see that one other user had the same problem in the past and I have tried the fixes suggested (not to use special characters) but it didn’t work. I use windows 11 Pro. See console below.

Please advise.

Kind regards

QGIS version: 3.22.5-Białowieża

QGIS code revision: c2723178

Qt version: 5.15.2

Python version: 3.9.5

GDAL version: 3.4.1

GEOS version: 3.10.2-CAPI-1.16.0

PROJ version: Rel. 8.2.1, January 1st, 2022

PDAL version: 2.3.0 (git-version: 9f35b7)

Algorithm started at: 2022-05-06T11:43:35

Algorithm ‘TrainImagesClassifier’ starting…

Input parameters:

{ ‘io.il’ : [‘C:/Users/daveh/Desktop/ORFEO Test DELETE/Stats/CarlitzdorpAloes20210818.tif’], ‘io.vd’ : [‘C:/Users/daveh/Documents/Aloes.shp’], ‘io.valid’ : None, ‘io.imstat’ : ‘C:\Users\daveh\Desktop\ORFEO Test DELETE\Stats\ImageStats.xml’, ‘io.out’ : ‘C:/Users/daveh/Desktop/ORFEO Test DELETE/Stats/KNN_Model.xml’, ‘io.confmatout’ : ‘C:/Users/daveh/Desktop/ORFEO Test DELETE/Stats/OutputKNN.xml’, ‘cleanup’ : False, ‘sample.mt’ : 1000, ‘sample.mv’ : 1000, ‘sample.bm’ : 1, ‘sample.vtr’ : 0.8, ‘sample.vfn’ : ‘Label’, ‘elev.dem’ : ‘’, ‘elev.geoid’ : ‘’, ‘elev.default’ : 0, ‘classifier’ : ‘knn’, ‘classifier.knn.k’ : 32, ‘rand’ : 0 }

2022-05-06 11:43:35 (INFO) TrainImagesClassifier: Default RAM limit for OTB is 256 MB

2022-05-06 11:43:35 (INFO) TrainImagesClassifier: GDAL maximum cache size is 1608 MB

2022-05-06 11:43:35 (INFO) TrainImagesClassifier: OTB will use at most 16 threads

2022-05-06 11:43:35 (INFO): Loading metadata from official product

2022-05-06 11:43:35 (INFO) TrainImagesClassifier: Polygon analysis…

2022-05-06 11:43:35 (INFO) TrainImagesClassifier: Elevation management: setting default height above ellipsoid to 0 meters

2022-05-06 11:43:35 (INFO): Estimated memory for full processing: 6478MB (avail.: 256 MB), optimal image partitioning: 26 blocks

2022-05-06 11:43:35 (INFO): Estimation will be performed in 27 blocks of 12090x651 pixels

Analyze polygons…: 100% [**************************************************] (0s)

2022-05-06 11:43:35 (INFO) TrainImagesClassifier: Sampling rates…

2022-05-06 11:43:35 (INFO) TrainImagesClassifier: Sampling strategy : fit the number of samples based on the smallest class

2022-05-06 11:43:35 (INFO) TrainImagesClassifier: Sampling rates for image 1 : className requiredSamples totalSamples rate

1 32876 32876 1

2 32876 42145 0.780069

2022-05-06 11:43:35 (INFO) TrainImagesClassifier: Sample selection…

2022-05-06 11:43:35 (INFO) TrainImagesClassifier: Elevation management: setting default height above ellipsoid to 0 meters

2022-05-06 11:43:35 (INFO) TrainImagesClassifier: Sampling strategy : set number of samples for each class

2022-05-06 11:43:35 (INFO) TrainImagesClassifier: Sampling rates : className requiredSamples totalSamples rate

1 32876 32876 1

2 32876 42145 0.780069

2022-05-06 11:43:35 (INFO): Estimated memory for full processing: 6478MB (avail.: 256 MB), optimal image partitioning: 26 blocks

2022-05-06 11:43:35 (INFO): Estimation will be performed in 35 blocks of 2864x2864 pixels

Selecting positions with periodic sampler…: 100% [**************************************************] (0s)

2022-05-06 11:43:36 (INFO) TrainImagesClassifier: Sample extraction…

2022-05-06 11:43:37 (INFO): Estimated memory for full processing: 9717.08MB (avail.: 256 MB), optimal image partitioning: 38 blocks

2022-05-06 11:43:37 (INFO): Estimation will be performed in 39 blocks of 12090x451 pixels

Extracting sample values…: 100% [**************************************************] (7s)

2022-05-06 11:43:44 (INFO): Estimated memory for full processing: 6478MB (avail.: 256 MB), optimal image partitioning: 26 blocks

2022-05-06 11:43:44 (INFO): Estimation will be performed in 35 blocks of 2864x2864 pixels

Split samples between training and validation…: 100% [**************************************************] (2s)

2022-05-06 11:43:47 (INFO) TrainImagesClassifier: Model training…

2022-05-06 11:43:47 (FATAL) TrainImagesClassifier: itk::ERROR: StatisticsXMLFileReader(00000202CEA71380): is a wrong Extension FileName : Expected .xml

Execution completed in 11.94 seconds

Results:

{‘io.confmatout’: 'C:/Users/daveh/Desktop/ORFEO Test ’
‘DELETE/Stats/OutputKNN.xml’,
‘io.out’: ‘C:/Users/daveh/Desktop/ORFEO Test DELETE/Stats/KNN_Model.xml’}

Loading resulting layers

Algorithm ‘TrainImagesClassifier’ finished

2

Dear @Sean,

Thank you for using OTB.

The QGIS plugin doesn’t support spaces in paths. You should try to rename the directory ORFEO Test DELETE to ORFEO_Test_DELETE.
Or you can try to put the path in quotes.

Best regards.
Julien :slight_smile:

3

Dear Julien

Thank you so much, that seems to have worked. It has taken months to figure this small glitch out.

Kind regards
Sean

1 Like
4

Hey Sean, I am having a little different error, I hope this is also peanuts for you:

QGIS version: 3.22.6-Białowieża

QGIS code revision: b6f33b1a

Qt version: 5.15.2

Python version: 3.9.5

GDAL version: 3.4.2

GEOS version: 3.10.2-CAPI-1.16.0

PROJ version: Rel. 9.0.0, March 1st, 2022

PDAL version: 2.3.0 (git-version: 1c17a7)

Algorithm started at: 2022-05-07T12:04:07

Algorithm ‘TrainImagesClassifier’ starting…

Input parameters:

{ ‘io.il’ : [‘C:/Users/31681/Documents/8.JJODATASCIENCE/1.BOMENBEHOUDVUGHT/2.Satellite Tree Guard/qgis/exp2_kaderingresult.tif’], ‘io.vd’ : [‘derle.shp’], ‘io.valid’ : None, ‘io.imstat’ : ‘C:\Users\31681\Documents\8.JJODATASCIENCE\1.BOMENBEHOUDVUGHT\2.Satellite Tree Guard\qgis\exp3_stats.xml’, ‘io.out’ : ‘C:/Users/31681/Documents/8.JJODATASCIENCE/1.BOMENBEHOUDVUGHT/2.Satellite Tree Guard/qgis/exp3_knnmodel.xml’, ‘io.confmatout’ : ‘C:/Users/31681/Documents/8.JJODATASCIENCE/1.BOMENBEHOUDVUGHT/2.Satellite Tree Guard/qgis/exp3_confmatrix.xml’, ‘cleanup’ : True, ‘sample.mt’ : 1000, ‘sample.mv’ : 1000, ‘sample.bm’ : 1, ‘sample.vtr’ : 0.8, ‘sample.vfn’ : ‘id’, ‘elev.dem’ : ‘’, ‘elev.geoid’ : ‘’, ‘elev.default’ : 0, ‘classifier’ : ‘knn’, ‘classifier.knn.k’ : 32, ‘rand’ : 0 }

2022-05-07 12:04:07 (INFO) TrainImagesClassifier: Default RAM limit for OTB is 256 MB

2022-05-07 12:04:07 (INFO) TrainImagesClassifier: GDAL maximum cache size is 789 MB

2022-05-07 12:04:07 (INFO) TrainImagesClassifier: OTB will use at most 16 threads

2022-05-07 12:04:08 (INFO): Loading metadata from official product

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Polygon analysis…

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Elevation management: setting default height above ellipsoid to 0 meters

2022-05-07 12:04:08 (INFO): Estimated memory for full processing: 45.2883MB (avail.: 256 MB), optimal image partitioning: 1 blocks

2022-05-07 12:04:08 (INFO): Estimation will be performed in 1 blocks of 1847x1074 pixels

Analyze polygons…: 100% [**************************************************] (0s)

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Sampling rates…

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Sampling strategy : fit the number of samples based on the smallest class

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Sampling rates for image 1 : className requiredSamples totalSamples rate

0 760 1495 0.508361

1 760 1699 0.447322

2 760 760 1

3 760 1248 0.608974

4 760 1153 0.65915

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Sample selection…

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Elevation management: setting default height above ellipsoid to 0 meters

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Sampling strategy : set number of samples for each class

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Sampling rates : className requiredSamples totalSamples rate

0 760 1495 0.508361

1 760 1699 0.447322

2 760 760 1

3 760 1248 0.608974

4 760 1153 0.65915

2022-05-07 12:04:08 (INFO): Estimated memory for full processing: 45.2883MB (avail.: 256 MB), optimal image partitioning: 1 blocks

2022-05-07 12:04:08 (INFO): Estimation will be performed in 2 blocks of 1408x1074 pixels

Selecting positions with periodic sampler…: 100% [**************************************************] (0s)

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Sample extraction…

2022-05-07 12:04:08 (INFO): Estimated memory for full processing: 67.9897MB (avail.: 256 MB), optimal image partitioning: 1 blocks

2022-05-07 12:04:08 (INFO): Estimation will be performed in 1 blocks of 1847x1074 pixels

Extracting sample values…: 100% [**************************************************] (0s)

2022-05-07 12:04:08 (INFO): Estimated memory for full processing: 45.2883MB (avail.: 256 MB), optimal image partitioning: 1 blocks

2022-05-07 12:04:08 (INFO): Estimation will be performed in 2 blocks of 1408x1074 pixels

Split samples between training and validation…: 100% [**************************************************] (0s)

2022-05-07 12:04:09 (INFO) TrainImagesClassifier: Model training…

2022-05-07 12:04:09 (FATAL) TrainImagesClassifier: itk::ERROR: StatisticsXMLFileReader(000001E976A155B0): .Satellite is a wrong Extension FileName : Expected .xml

Execution completed in 1.68 seconds

Results:

{‘io.confmatout’: 'C:/Users/31681/Documents/8.JJODATASCIENCE/1.BOMENBEHOUDVUGHT/2.Satellite ’
‘Tree Guard/qgis/exp3_confmatrix.xml’,
‘io.out’: 'C:/Users/31681/Documents/8.JJODATASCIENCE/1.BOMENBEHOUDVUGHT/2.Satellite ’
‘Tree Guard/qgis/exp3_knnmodel.xml’}

Loading resulting layers

Algorithm ‘TrainImagesClassifier’ finished

5

@julienosman Do you know what I am doing wrong here?

Thanks in advance!

QGIS version: 3.22.6-Białowieża

QGIS code revision: b6f33b1a

Qt version: 5.15.2

Python version: 3.9.5

GDAL version: 3.4.2

GEOS version: 3.10.2-CAPI-1.16.0

PROJ version: Rel. 9.0.0, March 1st, 2022

PDAL version: 2.3.0 (git-version: 1c17a7)

Algorithm started at: 2022-05-07T12:04:07

Algorithm ‘TrainImagesClassifier’ starting…

Input parameters:

{ ‘io.il’ : [‘C:/Users/31681/Documents/8.JJODATASCIENCE/1.BOMENBEHOUDVUGHT/2.Satellite Tree Guard/qgis/exp2_kaderingresult.tif’], ‘io.vd’ : [‘derle.shp’], ‘io.valid’ : None, ‘io.imstat’ : ‘C:\Users\31681\Documents\8.JJODATASCIENCE\1.BOMENBEHOUDVUGHT\2.Satellite Tree Guard\qgis\exp3_stats.xml’, ‘io.out’ : ‘C:/Users/31681/Documents/8.JJODATASCIENCE/1.BOMENBEHOUDVUGHT/2.Satellite Tree Guard/qgis/exp3_knnmodel.xml’, ‘io.confmatout’ : ‘C:/Users/31681/Documents/8.JJODATASCIENCE/1.BOMENBEHOUDVUGHT/2.Satellite Tree Guard/qgis/exp3_confmatrix.xml’, ‘cleanup’ : True, ‘sample.mt’ : 1000, ‘sample.mv’ : 1000, ‘sample.bm’ : 1, ‘sample.vtr’ : 0.8, ‘sample.vfn’ : ‘id’, ‘elev.dem’ : ‘’, ‘elev.geoid’ : ‘’, ‘elev.default’ : 0, ‘classifier’ : ‘knn’, ‘classifier.knn.k’ : 32, ‘rand’ : 0 }

2022-05-07 12:04:07 (INFO) TrainImagesClassifier: Default RAM limit for OTB is 256 MB

2022-05-07 12:04:07 (INFO) TrainImagesClassifier: GDAL maximum cache size is 789 MB

2022-05-07 12:04:07 (INFO) TrainImagesClassifier: OTB will use at most 16 threads

2022-05-07 12:04:08 (INFO): Loading metadata from official product

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Polygon analysis…

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Elevation management: setting default height above ellipsoid to 0 meters

2022-05-07 12:04:08 (INFO): Estimated memory for full processing: 45.2883MB (avail.: 256 MB), optimal image partitioning: 1 blocks

2022-05-07 12:04:08 (INFO): Estimation will be performed in 1 blocks of 1847x1074 pixels

Analyze polygons…: 100% [**************************************************] (0s)

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Sampling rates…

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Sampling strategy : fit the number of samples based on the smallest class

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Sampling rates for image 1 : className requiredSamples totalSamples rate

0 760 1495 0.508361

1 760 1699 0.447322

2 760 760 1

3 760 1248 0.608974

4 760 1153 0.65915

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Sample selection…

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Elevation management: setting default height above ellipsoid to 0 meters

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Sampling strategy : set number of samples for each class

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Sampling rates : className requiredSamples totalSamples rate

0 760 1495 0.508361

1 760 1699 0.447322

2 760 760 1

3 760 1248 0.608974

4 760 1153 0.65915

2022-05-07 12:04:08 (INFO): Estimated memory for full processing: 45.2883MB (avail.: 256 MB), optimal image partitioning: 1 blocks

2022-05-07 12:04:08 (INFO): Estimation will be performed in 2 blocks of 1408x1074 pixels

Selecting positions with periodic sampler…: 100% [**************************************************] (0s)

2022-05-07 12:04:08 (INFO) TrainImagesClassifier: Sample extraction…

2022-05-07 12:04:08 (INFO): Estimated memory for full processing: 67.9897MB (avail.: 256 MB), optimal image partitioning: 1 blocks

2022-05-07 12:04:08 (INFO): Estimation will be performed in 1 blocks of 1847x1074 pixels

Extracting sample values…: 100% [**************************************************] (0s)

2022-05-07 12:04:08 (INFO): Estimated memory for full processing: 45.2883MB (avail.: 256 MB), optimal image partitioning: 1 blocks

2022-05-07 12:04:08 (INFO): Estimation will be performed in 2 blocks of 1408x1074 pixels

Split samples between training and validation…: 100% [**************************************************] (0s)

2022-05-07 12:04:09 (INFO) TrainImagesClassifier: Model training…

2022-05-07 12:04:09 (FATAL) TrainImagesClassifier: itk::ERROR: StatisticsXMLFileReader(000001E976A155B0): .Satellite is a wrong Extension FileName : Expected .xml

Execution completed in 1.68 seconds

Results:

{‘io.confmatout’: 'C:/Users/31681/Documents/8.JJODATASCIENCE/1.BOMENBEHOUDVUGHT/2.Satellite ’
‘Tree Guard/qgis/exp3_confmatrix.xml’,
‘io.out’: 'C:/Users/31681/Documents/8.JJODATASCIENCE/1.BOMENBEHOUDVUGHT/2.Satellite ’
‘Tree Guard/qgis/exp3_knnmodel.xml’}

Loading resulting layers

Algorithm ‘TrainImagesClassifier’ finished

6

Hi Julien

I was wondering if it would be possible to locate certain objects in drone maps using OTB, in particular certain vegetation for forestry (for tree counting and canopy area). Cloud based AI/machine learning platforms like Picterra do the job of identifying, counting and polygonizing pines in pine plantations, can OTB be used to do the same thing? Or specific vegetation types in wild forests?

Thank you

7

Hi jorritxl

Can you provide some background?

Thanks
Sean Hill

8

I have never tried this specific usage of OTB, but I don’t see why it wouldn’t be possible. Considering a correct training dataset, you should be able to use the Machine Leaning algorithms available in OTB to train a model that will classify the trees in your maps.

9

Thank you Julien

Should someone else ask (considering drones are rapidly becoming useful tools in conservation, agriculture, forestry and disaster management), I found this paper that may be of use for those using extremely high resolution drone orthomosaic maps and terrain models:

Drone image processing with Orfeo Toolbox

I have not figured it out yet, but I’m sure some of the more clued up users of Orfeo may find it useful. Should you find the time, would you be able to read through it and give some advice/notes/YouTube tutorial? I think there would be a decent number of GIS users that utilize drones as remote sensors that would greatly appreciate it.

Kind regards
Sean

1 Like