Setup python to get the orfeo toolbox work properly together with pyqgis from a pycharm ide or a standalone python script

I am changing my python environment from working with ArcGis and Arcpy to working with QGIS and PyQgis. And I also want to use the machinelearning possibilities from the orfeo toolbox. In doing so
I have succesfully installed the orfeo toolbox (7.0.0) in my QGIS 3.13 installation on my Win10 PC.

Now I want to use stand alone scripts or aan IDE like pycharm to access the orfeo toolbox like other pyqgis processing tools like this:

processing.run("otb:TrainImagesClassifier.......

the error message I receive is:

core.QgsProcessingException: Error: Algorithm otb:TrainImagesClassifier not found.

also the simple command:

import otbApplication as otb

in python results in:

ModuleNotFoundError: No module named 'otbApplication'

How do I have to setup python to get the orfeo toolbox work properly together with pyqgis?

I am using the python 3.7 interpreter from QGIS
I am using the PyCharm ide and try to set things up using:

@echo off
SET OSGEO4W_ROOT=C:\OSGeo4W64
call "%OSGEO4W_ROOT%"\OTB-7.0.0-Win64\otbenv.bat
call "%OSGEO4W_ROOT%"\bin\o4w_env.bat
call "%OSGEO4W_ROOT%\bin\qt5_env.bat"
call "%OSGEO4W_ROOT%\bin\py3_env.bat"
call "%OSGEO4W_ROOT%"\apps\grass\grass78\etc\env.bat
@echo off
path %PATH%;%OSGEO4W_ROOT%\apps\qgis\bin
path %PATH%;%OSGEO4W_ROOT%\apps\grass\grass78\lib
path %PATH%;%OSGEO4W_ROOT%\apps\Qt5\bin
path %PATH%;%OSGEO4W_ROOT%\apps\Python37\Scripts
path %PATH%;C:\qgis-plugins\qgis-otb-plugin
path %PATH%;%OSGEO4W_ROOT%\apps\qgis\python\plugins
path %PATH%;%OSGEO4W_ROOT%\apps\qgis\python\plugins\processing
path %PATH%;C:\Users\...\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins
path %PATH%;%OSGEO4W_ROOT%\apps\saga
path %PATH%;%OSGEO4W_ROOT%\apps\gdal2

set GDAL_FILENAME_IS_UTF8=YES

set VSI_CACHE=TRUE
set VSI_CACHE_SIZE=1000000
set QT_PLUGIN_PATH=%OSGEO4W_ROOT%\apps\qgis\qtplugins;%OSGEO4W_ROOT%\apps\qt5\plugins

set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\qgis\python
set PYTHONHOME=%OSGEO4W_ROOT%\apps\Python37

set QT_QPA_PLATFORM_PLUGIN_PATH=%OSGEO4W_ROOT%\apps\Qt5\plugins\platforms
set QGIS_PREFIX_PATH=%OSGEO4W_ROOT%\apps\qgis
SET PYCHARM="C:\Program Files\JetBrains\PyCharm Community Edition 2020.1.1\bin\pycharm64.exe"
start "PyCharm aware of Quantum GIS" /B %PYCHARM% %*

cmd.exe

I hope you can give me some hints where my thinking goes wrong.

Hi,

  1. To use QGIS processing API, I think that you must tell QGIS where is installed OTB.
  2. To use OTB python API (which is not the same thing as the QGIS API), you could define the following environment variables:
  • PATH <-- otb bin directory
  • LD_LIBRARY_PATH (PATH on windows?) <-- otb lib directory
  • OTB_APPLICATION_PATH <-- otb lib/otb/applications directory
  • PYTHONPATH <-- otb lib/otb/python directory

Hope that helps you!

Hi @remi.cresson,

Thank you very much for your answer. It got me one step further, I think. Instead of

`PYTHONPATH`  <-- otb  `lib/otb/python`  directory

I used:

`PYTHONPATH`  <-- otb  `lib/python`  directory

because the lib/otb/python directory does not exist. And the file otbApllication.py is in the otblib/python directory.

The result is the following setup (where changes following your advice are marked:

@echo off
SET OSGEO4W_ROOT=C:\OSGeo4W64
call "%OSGEO4W_ROOT%"\OTB-7.0.0-Win64\otbenv.bat
call "%OSGEO4W_ROOT%"\bin\o4w_env.bat
call "%OSGEO4W_ROOT%\bin\qt5_env.bat"
call "%OSGEO4W_ROOT%\bin\py3_env.bat"
call "%OSGEO4W_ROOT%"\apps\grass\grass78\etc\env.bat
@echo off
path %PATH%;%OSGEO4W_ROOT%\apps\qgis\bin
path %PATH%;%OSGEO4W_ROOT%\apps\grass\grass78\lib
path %PATH%;%OSGEO4W_ROOT%\apps\Qt5\bin
path %PATH%;%OSGEO4W_ROOT%\apps\Python37\Scripts
**path %PATH%;%OSGEO4W_ROOT%\OTB-7.0.0-Win64\bin**
**path %PATH%;%OSGEO4W_ROOT%\OTB-7.0.0-Win64\lib**
path %PATH%;%OSGEO4W_ROOT%\apps\qgis\python\plugins
path %PATH%;%OSGEO4W_ROOT%\apps\qgis\python\plugins\processing
path %PATH%;C:\Users\...\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins
path %PATH%;%OSGEO4W_ROOT%\apps\saga
path %PATH%;%OSGEO4W_ROOT%\apps\gdal2

set GDAL_FILENAME_IS_UTF8=YES

set VSI_CACHE=TRUE
set VSI_CACHE_SIZE=1000000
set QT_PLUGIN_PATH=%OSGEO4W_ROOT%\apps\qgis\qtplugins;%OSGEO4W_ROOT%\apps\qt5\plugins

set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\qgis\python;**%OSGEO4W_ROOT%\OTB-7.0.0-Win64\lib\python**
set PYTHONHOME=%OSGEO4W_ROOT%\apps\Python37
**set OTB_APPLICATION_PATH=%OSGEO4W_ROOT%\OTB-7.0.0-Win64\lib\otb\applications**


set QT_QPA_PLATFORM_PLUGIN_PATH=%OSGEO4W_ROOT%\apps\Qt5\plugins\platforms
set QGIS_PREFIX_PATH=%OSGEO4W_ROOT%\apps\qgis
SET PYCHARM="C:\Program Files\JetBrains\PyCharm Community Edition 2020.1.1\bin\pycharm64.exe"
start "PyCharm aware of Quantum GIS" /B %PYCHARM% %*

cmd.exe

When I try to import otbApplication:

import otbApplication as otb

I receive the following error:

Traceback (most recent call last):
  File "C:\OSGEO4~1\OTB-7.0.0-Win64\lib\python\otbApplication.py", line 14, in swig_import_helper
    return importlib.import_module(mname)
  File "C:\OSGEO4~1\apps\Python37\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 670, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 583, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1043, in create_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: DLL load failed: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/D.J/panels/test1.py", line 1, in <module>
    import otbApplication as otb
  File "C:\OSGEO4~1\OTB-7.0.0-Win64\lib\python\otbApplication.py", line 17, in <module>
    _otbApplication = swig_import_helper()
  File "C:\OSGEO4~1\OTB-7.0.0-Win64\lib\python\otbApplication.py", line 16, in swig_import_helper
    return importlib.import_module('_otbApplication')
  File "C:\OSGEO4~1\apps\Python37\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: DLL load failed: The specified module could not be found.

Do you have an idea what causes this error?

Regards!

Hi,
I did not see that at first:

I believe that this should init properly the env vars for OTB.
However refering to the doc, it looks like you must also call the otbenv script to init the python bindings for OTB. Can you try that?
Take a look in those scripts, you will see what env vars you need to declare.

Hi @remi.cresson,

I started (again) from scratch using this post.

@echo off
SET OSGEO4W_ROOT=C:\OSGeo4W64

call "%~dp0\o4w_env.bat"
call "%OSGEO4W_ROOT%"\apps\grass\grass78\etc\env.bat
call qt5_env.bat
call py3_env.bat

@echo off
path %OSGEO4W_ROOT%\apps\qgis\bin;%OSGEO4W_ROOT%\apps\grass\grass78\lib;%OSGEO4W_ROOT%\apps\grass\grass78\bin;%PATH%
set QGIS_PREFIX_PATH=%OSGEO4W_ROOT:\=/%/apps/qgis
set GDAL_FILENAME_IS_UTF8=YES
rem Set VSI cache to be used as buffer, see #6448
set VSI_CACHE=TRUE
set VSI_CACHE_SIZE=1000000
set QT_PLUGIN_PATH=%OSGEO4W_ROOT%\apps\qgis\qtplugins;%OSGEO4W_ROOT%\apps\qt5\plugins
SET PYCHARM="C:\Program Files\JetBrains\PyCharm Community Edition 2020.1.1\bin\pycharm64.exe"
start "PyCharm aware of Quantum GIS" /B %PYCHARM% %*

When I add “call “%OSGEO4W_ROOT%”\OTB-7.1.0-Win64\otbenv.bat”, then the following error occurs

Traceback (most recent call last):
  File "C:/Users/D.J/panels/test.py", line 4, in <module>
    from qgis.core import QgsApplication, QgsProcessingFeedback
  File "C:\OSGeo4W64\apps\qgis\python\qgis\core\__init__.py", line 25, in <module>
    from qgis._core import *
ImportError: DLL load failed: Kan opgegeven procedure niet vinden.

When I looked into otbenv.bat I could not quite understand everything that was in there, but t seems that three settings had to be set:

call :prefix_path PATH "%CURRENT_SCRIPT_DIR%bin"
call :prefix_path PYTHONPATH "%CURRENT_SCRIPT_DIR%lib\python"
call :prefix_path OTB_APPLICATION_PATH "%CURRENT_SCRIPT_DIR%lib\otb\applications"

I translated that into:

path %PATH%;%OSGEO4W_ROOT%\OTB-7.1.0-Win64\bin
path %PYTHONPATH%;%OSGEO4W_ROOT%\OTB-7.1.0-Win64\lib\python
path %OTB_APPLICATION_PATH%;%OSGEO4W_ROOT%\OTB-7.1.0-Win64\lib\otb\applications

I inserted that into my .bat file, and it got me the same error.

It seems that the OTB settings are fighting the QGIS settings.

Hi @Ianuam and @remi.cresson,

I’m currently getting pretty much the same errors as you do when using otb 7.1.0 in combination with python 3.7.
Switching to an environment with python 3.5 solved the errors for me. However, using 3.5 creates some complications with other packages that I need to use… so it’s definitely not an optimal solution.
If you have found a solution for fixing the problem in 3.7 I’ll gladly hear it.

Hi @LoutK,

I am pleased that I am not the only one struggling.
I succeeded in setting up a 3.5 environment with OTB, but then I lost the cgis.core.
Were you able to access both packages within the python 3.5 environment?

Best