By testing OTB 9.0.0 I hit an issue with our current processing chains.
We are using extended filenames everywhere (great concept to isolate options on a per-file basis), and given the syntax I treated it as an URL. I then used the following format in CLI:
/data/test.tif?gdal:co:TILED=YES # KO
However, when updating to 9.0.0, I ran into an “unsupported format” error. I’m attaching the test I did with the official docker images ( otb9_extfn_test.txt ). A college convinced me to test with the official method by adding the “&” before the argument, and indeed it worked !
/data/test.tif?&gdal:co:TILED=YES # OK
The question is: is it a bug or a feature ? I can hear this can be a problem if someone wants to use question marks in their filenames, and the “?&” pair is safer. However, I find it unfortunate to have something so close to the URL spec but for this leading “&”. And also to be forced to add that symbol in all my scripts .
To my knowledge the syntax has always been the same with “?&” , I checked on the code there was no change on the extendedfilenameHelper. This is weird it used to work with “?” only. What version were you using before ?
As stated by the cookbook the official way to use extended filename is by using “?&”
I don’t know what changed, but you can test it yourself with the docker example in my first post: it was working for 8.1.2 and less, but not for 9.0.0.
I’ve been using OTB for over 10 years with this syntax. By the way, thanks for the good work!
The doc is well-formed, no worry. But my question is: is this an intentional change ? Until now, it was possible to use it as an URL query. By following this syntax, it looks like we need to define an empty key/value pair before starting. I like to view paths as a subset of the URL specification, and I thought of extended-filename as a natural expression of that view. If the change was not planned, can you reverse it ?
I also wanted to raise this issue for anybody using the same syntax, as the error message is not obvious.
In the implementation before 9.0 there was a problem when you use multiple key value pairs in an extended filename. With this implementation it permits to handle that case but yes, it indeed force you to modify your scripts to be compatible with this type of extended filename writing. So, that was an intentional change to robustify the syntax.
Maybe the documentation is not as clear as we see it ? Don’t hesitate to create an issue on the gitlab if you have some tips on how to improve that documentation about the extended filename
Thank you, that answers my question: I will modify my scripts!
But as a suggestion, it is also possible to merge the “URL” arguments with the “extended” ones, eg. by using a namespace for the URL arguments (which are then passed through to the “reader”). Or to split with the last question mark.
A “filename” with two question marks is confusing from my point of view. Moreover, the “?&” string is often accepted as legit in regular URL, which can lead to further confusion if used twice.
Thanks for your suggestions, can you open an issue on the gitlab ? It would be very interesting to have a discussion on how to implement it for the next OTB version and see with the PSC their interest in this change
.