No doc on ObjectsRadiometricStatistics

alobo · January 31, 2019, 12:20pm

Just to note that, AFAIK, there is no mention of ObjectsRadiometricStatistics in the docs. Once you know
it exists, it is simple to use -help option, but you must know about its existence first…
Agus

Antoine · January 31, 2019, 12:28pm

Hi Agus,
That would be normal as there is no such application in otb!
The application you are talking about is coming from a remote module : https://github.com/gpo-geo/SertitObject

alobo · January 31, 2019, 12:32pm

What am I using then?

alobo@Delia:~/OTB-6.6.0-Linux64/bin$ ls -l otbcli_Objects*
-rwxr-xr-x 1 alobo alobo 382 Jun 20  2018 otbcli_ObjectsRadiometricStatistics
alobo@Delia:~/OTB-6.6.0-Linux64/bin$ otbcli_ObjectsRadiometricStatistics -help

This is the ObjectsRadiometricStatistics (ObjectsRadiometricStatistics) application, version 6.6.0

Compute features attributes of a vector dataset over an image.
Tags: FeatureExtraction Radiometry 

This application computes radiometric and shapes attributes on a vector dataset, using an image. The results are stored in the attribute table. Shape attributes are : number of pixels, flatness, roundness, elongation, perimeter. Radiometric attributes are for each band of the input image : mean, standard-deviation, median, variance, kurtosis, skewness.

Antoine · January 31, 2019, 12:36pm

What did you not understand in my previous answer?

alobo · January 31, 2019, 12:38pm

What did you not understand from mine…
(and we could get into an infinite loop, so I just break it here)

Antoine · January 31, 2019, 12:41pm

What are you using?
An application coming from a remote module.
Why no official documentation for this application?
Because it is not part of OTB but from a remote module.
Is this enough? Or do you have other “infinite loop” that I could clarify?

alobo · January 31, 2019, 12:48pm

No, obviously not enough. Users need documentation on all available applications.
At least they should be mentioned, along with what they do. If you cannot maintain
the details, include a link and advise the user to use the -h option
User-unfriendly attitudes are at the bottom of why excellent tools do not get the popularity that they deserve.

jmichel · February 1, 2019, 8:11am

Here is what a kind and friendly answer may look like (users are not supposed to know every internal details of OTB by heart to be allowed to ask a question):

@alobo The reason why there is no documentation for this application is that it comes from a remote module ( https://github.com/gpo-geo/SertitObject ) that is not developed and maintained by the otb development team, but only shipped in binary packages. Shipping remote modules in binary packages is a service we offer to remote modules developers so that they can reach users more easily.

I understand that from a user perspective this is confusing as there is no clear way to know which applications come from remote modules. Maybe you can file an issue on this topic in gitlab, so that we can try working out a solution for this problem ?

In the mean time, do not hesitate to report the issue to the remote module github repository.

poughov · February 1, 2019, 10:38am

Thanks for reporting this @alobo.

I’ve opened issue https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/issues/1832 to track improvements about this.

alobo · February 1, 2019, 11:45am

The “unfriendliness” that is of concern is not the one that could derive from the message. That, in any case, would affect only 1 person.
The real concern is that such a precious gem as ObjectRadiometricStatistics is not shown to the eyes of all OTB users. This tool is not a low-level internal function, is an otbcli_ application at the same level than the rest and one of much interest to many OTB users.
ObjectRadiometricStatistics performs a task that is of general interest for the analysis of multispectral imagery, and does so at a considerable speed (10x faster than the equivalent R extract() function, albeit a bit less general). Being able to calculate the statistics of a multi-spectral image by the levels of a categorical spatial object (aka zonal statistics) is a very important tool on its own, thus users should know that they have an efficient implementation available in OTB (they do not in QGIS, for example, where this operation is performed for 1-layer rasters only).
Also, I would say that the fact that such an application is not developed and maintained by the otb team, is precisely one reason to declare that is being used, specify what it does, the author and the link. I understand you could not maintain its doc, but you can certainly keep the link.

jmichel · February 1, 2019, 12:09pm

@alobo did you by any chance give a try to the new ZonalStastitics app in otb develop branch ( see https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/merge_requests/222 ) ?

Pros :

It is an official app. If the doc has to be updated, we’ll fix it quickly,
It uses streaming to compute the stats, whereas ObjectsRadiometricStatistics loads the full data in memory
It covers more cases (vector/raster as input/output)

Cons:

It computes less statistics. Extending the set of statistics is complex, because of the streaming. Nevertheless, something could be done by estimating histogram by polygon and then derive some stats from it (see discussion in MR).

You should be able to give it a try with nightly packages: https://www.orfeo-toolbox.org/packages/nightly/latest/

alobo · February 1, 2019, 1:05pm

I was not aware of such development, good to know!
Among the statistics, I miss median and median absolute deviation (the robust analogs of mean and std dev. respectively).
What you mention about the histogram would be a solution, but avoid doing it at the expense of speed. Perhaps we could have 2 tools: one very fast, intended for many polygons (e.g. result of segmentation) and another one, slower but with more statistics, for polygons derived from maps or areas of interest, where more statistics could be calculated.

I’ve downloaded and installed
https://www.orfeo-toolbox.org/packages/nightly/latest/OTB-6.6.1-Linux64.run
but get an error

/home/alobo/OTB-6.6.1-Linux64/bin/otbApplicationLauncherCommandLine: symbol lookup error: /home/alobo/OTB-6.6.1-Linux64/bin/otbApplicationLauncherCommandLine: undefined symbol: _ZN3otb20ConfigurationManager17InitOpenMPThreadsEv

what would be the otbcli command? otbcli_ZonalStatistics ?

remi.cresson · February 1, 2019, 1:54pm

Yes. I am not familiar with all the metrics computed in ObjectsRadiometricStatistics, but I think that some of them can not be computed in a streamable way… that’s why we will probably keep 2 applications in the future.
However, as @jmichel say there is room for improvement of the ZonalStatistics application, that would not affect its processing speed.

Antoine · February 4, 2019, 5:09pm

@alobo you need to download packages of the 6.7 version to get ZonalStatistic.You can find it here. Keep in mind that those packages might be buggy as they are not released yet!

However you seem to have a problem with 6.6.1… Do you have OpenMP installed on your linux?

alobo · February 5, 2019, 9:46am

Do you have OpenMP installed on your linux

Not on my Debian desktop, perhaps on the cluster (I have asked).
There is no mention (afaik) on OpenMP being a requirement for OTB, actually, is not everything included in the *.run file?
Should I install OpenMP?
Thanks

Antoine · February 5, 2019, 10:07am

Yes there is no requirement on OpenMP it has been introduced in 6.6.1, and is in the packages (.run) but…
Can you post the output of:

ldd otbApplicationLauncherCommandLine | grep libgomp

Just to make sure that nothing went wrong during the installation.
Then the result of:

$ ldd bin/otbApplicationLauncherCommandLine | grep OTBCommon
$ objdump -t lib/libOTBCommon-6.6.so | grep OpenMP

I hope it will tell us something…
Oh! And also… What is your Debian?

alobo · February 6, 2019, 2:36pm

Here:

alobo@Delia:~$ . /home/alobo/OTB-6.6.1-Linux64/otbenv.profile
alobo@Delia:~$ $PATH
bash: /home/alobo/OTB-6.6.1-Linux64/bin:/home/alobo/OTB-6.6.1-Linux64/bin:/home/alobo/OTB-6.6.1-Linux64/bin:/home/alobo/OTB-6.6.1-Linux64/bin:/home/alobo/seadas-7.5.1/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games: No such file or directory
alobo@Delia:~$ ldd otbApplicationLauncherCommandLine | grep libgomp
ldd: ./otbApplicationLauncherCommandLine: No such file or directory
alobo@Delia:~$ ldd /home/alobo/OTB-6.6.1-Linux64/bin/otbApplicationLauncherCommandLine | grep libgomp
	libgomp.so.1 => /home/alobo/OTB-6.6.1-Linux64/lib/libgomp.so.1 (0x00000036bba00000)
alobo@Delia:~$ ldd /home/alobo/OTB-6.6.1-Linux64/bin/otbApplicationLauncherCommandLine | grep OTBCommon
	libOTBCommon-6.6.so.1 => /home/alobo/OTB-6.6.0-Linux64/lib/libOTBCommon-6.6.so.1 (0x00007f387f577000)
alobo@Delia:~$ objdump -t /home/alobo/OTB-6.6.1-Linux64/lib/libOTBCommon-6.6.so | grep OpenMP
0000000000010a30 g     F .text	0000000000000013              _ZN3otb20ConfigurationManager17InitOpenMPThreadsEv

Thanks!

Antoine · February 6, 2019, 5:52pm

Ok, that is weird… What is your Debian version? I’ll try to reproduce the error in a docker.

alobo · February 8, 2019, 9:48am

6-6-0 and previous versions worked fine.
My system:
alobo@Delia:~$ uname -a

Linux Delia 4.16.0-2-amd64 #1 SMP Debian 4.16.12-1 (2018-05-27) x86_64 GNU/Linux
alobo@Delia:~$  lsb_release -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux testing (buster)
Release:	testing
Codename:	buster
alobo@Delia:~$ cat /etc/debian_version
buster/sid