Hello
I’ve noticed that the current GDAL version supports thread-safe reading of multiple files. Is there a plan to incorporate this feature in future versions of OTB for parallel file reading? Is parallel reading already implemented through another method (for TIFF files, for instance)?
I often observe sequential activity when processing a large number of input files, likely due to sequential file-by-file decompression.
Thanks !
Hello
I’ve identified a bottleneck in my code, which was the number of threads actually being used by OTB (in this case, just one). From my reading of the code, I found that the number of threads is RAM-driven, and ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS only sets a maximum limit (and defaults to the number of virtual cores). For programs that use multiple filters connected together, it’s advisable to increase the default RAM allocation in order to use the available CPU cores (via OTB_MAX_RAM_HINT).
It would be helpful to display the actual number of threads used in the log, along with the number of blocks. I had to use a printf inside my filter to determine that number.
My previous question still stands, as it is common to observe periods of high computation followed by downtimes where the main thread is at 100%.
Thanks !