Hello everyone !
I am working on an object-based classification from one false-color image. In order to limit error of classification of my three categories (Artificial surface, tree vegetation and herbaceous vegetation), I created 5 classifications to fusion them by a majority voting (FusionOfClassification algorithm).
I divided my training vector into 5 subsets : for each classification model 4 subsets were used as a training vector and the lastest was used as a validating vector. Thus each subset was used as a validating vector.
I also removed shadow segments before classifications (segments showing a mean value of brightness less than 70).
Each of my classification model (from TrainVectorClassifier algorithm) shows good results (about 0.95 for precision, Recall or F-Score) obtained in the “log” window.
However, I wanted to test my final classification (Fusion of the 5 classif by majority voting) with ComputeConfusionMatrix algorithm and I used a new subset from all my training vector data. I otbained metrics around 0.75 for F-Score, Precision, Recall for each of the 3 categories.
I think my final classification is good, metrics should be rather high as my classification models (0.95 / 1 for F-Score, Precision, Recall) but its evaluation shows poor results (0.75 for each metrics).
So my question is, how evaluate a classification obtained by a majority voting ? Does it make sense to use the ComputeConfusionMatrix agorithm on a classification fusionned by majority voting?
Do I have to average the metrics for each of the models to get an assessment of the final classification? Or is my validating vector the problem for the evaluation of the final classification?
Thank you in advance for your answer.