2017-12-18 23:18 CET

View Issue Details Jump to Notes ]
IDProjectCategoryView StatusLast Update
0001365Orfeo Toolbox (OTB)Generalpublic2017-04-05 10:21
Reportervfdev-5 
Assigned Togpasero 
PrioritynormalSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
Platformx64OSMacOSXOS VersionEl Captain 10.11
Summary0001365: Python wrapper OTB-5.8.0: @rpath problem on MacOSX El Captain 10.11
Description
After environment configuration : PATH, OTB_OTB_APPLICATION_PATH. Import of otbApplication in Python ends up with :

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/otbApplication.py", line 28, in <module>
    _otbApplication = swig_import_helper()
  File "/Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/otbApplication.py", line 24, in swig_import_helper
    _mod = imp.load_module('_otbApplication', fp, pathname, description)
ImportError: dlopen(/Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/_otbApplication.so, 2): Library not loaded: @rpath/libOTBApplicationEngine-5.10.1.dylib
  Referenced from: /Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/_otbApplication.so
  Reason: image not found
```

Reason is problably in with newer MacOSX policies. Maybe, `-rpath` option need to be specified during compilation. See http://stackoverflow.com/questions/14656657/linking-a-dynamic-library-libjvm-dylib-in-mac-os-x-rpath-issue#14657625
TagsNo tags attached.
Attached Files

-Relationships
+Relationships

-Notes

~0004045

rashadkm (developer)

Last edited: 2017-02-24 15:13

View 2 revisions

you need to set pythonpath not the path.

I think there is an otbenv.profile in the package.

rpath is well set on compilation and on installation. Yes in both stages we take advantage of rpath!

i think you are missing only PYTHONPATH here.

~0004046

vfdev-5 (reporter)

Last edited: 2017-02-24 21:36

View 2 revisions

Here is the log with otbenv.profile sourced :

```
USER:Documents user$ cd OTB-5.10.0-Darwin64/
USER:OTB-5.10.0-Darwin64 user$ ls
LICENSE Monteverdi.app bin otbenv.profile
Mapla.app README lib share
USER:OTB-5.10.0-Darwin64 user$ source otbenv.profile
USER:OTB-5.10.0-Darwin64 user$ python
Python 2.7.13 (default, Dec 17 2016, 23:03:43)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import otbApplication
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/otbApplication.py", line 28, in <module>
    _otbApplication = swig_import_helper()
  File "/Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/otbApplication.py", line 24, in swig_import_helper
    _mod = imp.load_module('_otbApplication', fp, pathname, description)
ImportError: dlopen(/Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/_otbApplication.so, 2): Library not loaded: @rpath/libOTBApplicationEngine-5.10.1.dylib
  Referenced from: /Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/_otbApplication.so
  Reason: image not found
>>>
```

Same in 5.8.0 :

```
USER:OTB-5.8.0-Darwin64 user$ ls
LICENSE Monteverdi.app bin otbenv.profile
Mapla.app README lib share
USER:OTB-5.8.0-Darwin64 user$ source otbenv.profile
USER:OTB-5.8.0-Darwin64 user$ python
Python 2.7.13 (default, Dec 17 2016, 23:03:43)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import otbApplication
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/USER/Documents/OTB-5.8.0-Darwin64/lib/python/otbApplication.py", line 28, in <module>
    _otbApplication = swig_import_helper()
  File "/Users/USER/Documents/OTB-5.8.0-Darwin64/lib/python/otbApplication.py", line 24, in swig_import_helper
    _mod = imp.load_module('_otbApplication', fp, pathname, description)
ImportError: dlopen(/Users/USER/Documents/OTB-5.8.0-Darwin64/lib/python/_otbApplication.so, 2): Library not loaded: @rpath/libOTBApplicationEngine-5.8.1.dylib
  Referenced from: /Users/USER/Documents/OTB-5.8.0-Darwin64/lib/python/_otbApplication.so
  Reason: image not found
>>>
```

~0004050

gpasero (administrator)

I can't reproduce the bug on MacOS X 10.10 (the platform used to build packages).

There is maybe something with @rpath usage, but I would need a more recent platform to investigate.

~0004051

gpasero (administrator)

As a simple test, could you try to :
* move the file /Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/_otbApplication.so to its parent directory
* add this parent directory (/Users/.../OTB-5.10.0-Darwin64/lib) to PYTHONPATH in otbenv.profile
* try again to import otbApplication from python

~0004052

grizonnetm (administrator)

Could it be related to the new security protection introduced in El Capitan?

https://en.wikipedia.org/wiki/System_Integrity_Protection

@vfdev-5 can you try to disable temporarily the system protection and try again otb binaries to see if it's related?

http://osxdaily.com/2015/10/05/disable-rootless-system-integrity-protection-mac-os-x/

It cuold help us to narrow down the problem.

Thanks in advance.

~0004053

vfdev-5 (reporter)

Last edited: 2017-02-28 20:21

View 2 revisions

Yes, it is related to SIP.

Unfortunately, I can not try to disable this. The problem is related with @rpath setting.

For example, the following commands help me to overcome it. Idea is to remplace @rpath by relative location with `install_name_tool`:

```
otool -L _otbApplication.so | grep -ohe "@rpath/\(\S\)*" | grep -ohe "lib\(\S\)*" | while read i; do install_name_tool -change @rpath/$i $PWD/lib/$i _otbApplication.so; done
```
and for those who use Python from brew and not system one:
```
install_name_tool -change /System/Library/Frameworks/Python.framework/Versions/2.7/Python /usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/Python _otbApplication.so
```

https://gist.github.com/vfdev-5/7d958b381e3795d96e9b0a6d7609f17b

~0004054

linkalis (reporter)

I am experiencing the same issue on Mac OS Sierra version 10.12.3. I was able to disable my System Integrity Protection so that when I run `csrutil status`, I now see "System Integrity Protection status: disabled" on my Mac.

I then deleted my previous extracted OTB-5.10.0-Darwin64 folder, and re-extracted a new version of it from the OTB-5.10.0-Darwin64.run file. Then, starting from (what I hope amounts to) a "fresh" install, I'm still seeing the error:

```
ImportError Traceback (most recent call last)
<ipython-input-1-c224f4bb32fe> in <module>()
      3
      4 import numpy
----> 5 import otbApplication

/Users/USER/OrfeoToolbox/OTB-5.10.0-Darwin64/lib/python/otbApplication.py in <module>()
     26 fp.close()
     27 return _mod
---> 28 _otbApplication = swig_import_helper()
     29 del swig_import_helper
     30 else:

/Users/USER/OrfeoToolbox/OTB-5.10.0-Darwin64/lib/python/otbApplication.py in swig_import_helper()
     22 if fp is not None:
     23 try:
---> 24 _mod = imp.load_module('_otbApplication', fp, pathname, description)
     25 finally:
     26 fp.close()

ImportError: dlopen(/Users/USER/OrfeoToolbox/OTB-5.10.0-Darwin64/lib/python/_otbApplication.so, 2): Library not loaded: @rpath/libOTBApplicationEngine-5.10.1.dylib
  Referenced from: /Users/USER/OrfeoToolbox/OTB-5.10.0-Darwin64/lib/python/_otbApplication.so
  Reason: image not found
```

If it makes a difference, I'm trying to run this via an Anaconda Python 2.7 virtual environment. (I installed the Anaconda 3.6 package as my base version, and have created a separate virtual environment with Python 2.7 that I'm hoping to use for Orfeo Toolbox.) I have a file called orfeo_toolbox.pth located in my virtual environment's python2.7/site-packages folder that points to the /Users/USER/OrfeoToolbox/OTB-5.10.0-Darwin64/lib/python folder of my Orfeo install. The path seems to be detected just fine when I try to import otbApplication, because it's able to detect the otbApplication.py module. It just errors out on line 24 of the module when it tries to import the .dylib files.

I have not modified anything in the otbenv.profile file.

~0004055

gpasero (administrator)

I found the root of this issue and I could reproduce it on the platform used to build the package (MacOS X 10.10).

During package deployment, the LC_RPATH load command is added to all the executables and libraries : except on _otbApplication.so
This LC_RPATH contains the path to package lib's folder.

~0004057

gpasero (administrator)

I pushed a fix : https://git.orfeo-toolbox.org/otb.git/commit/442f70c431f78596f42a39bc40925c5f7cabcea9

The library _otbApplication.so was missing a LC_RPATH load command.

~0004064

gpasero (administrator)

Fix working fine.

~0004065

gpasero (administrator)

My thanks to the users who reported the issue. If you want to test MacOS X packages after the fix, you can download latest nightly builds at : https://www.orfeo-toolbox.org/packages/nightly/latest/
+Notes

-Issue History
Date Modified Username Field Change
2017-02-22 20:49 vfdev-5 New Issue
2017-02-24 15:08 rashadkm Note Added: 0004045
2017-02-24 15:13 rashadkm Note Edited: 0004045 View Revisions
2017-02-24 21:35 vfdev-5 Note Added: 0004046
2017-02-24 21:36 vfdev-5 Note Edited: 0004046 View Revisions
2017-02-28 14:36 gpasero Note Added: 0004050
2017-02-28 14:36 gpasero Assigned To => gpasero
2017-02-28 14:36 gpasero Status new => acknowledged
2017-02-28 14:51 gpasero Note Added: 0004051
2017-02-28 14:54 grizonnetm Note Added: 0004052
2017-02-28 19:39 vfdev-5 Note Added: 0004053
2017-02-28 20:21 vfdev-5 Note Edited: 0004053 View Revisions
2017-03-01 07:02 linkalis Note Added: 0004054
2017-03-01 17:55 gpasero Note Added: 0004055
2017-03-01 17:55 gpasero Status acknowledged => confirmed
2017-03-02 16:34 gpasero Note Added: 0004057
2017-03-02 16:34 gpasero Status confirmed => assigned
2017-03-03 12:09 gpasero Note Added: 0004064
2017-03-03 12:09 gpasero Status assigned => resolved
2017-03-03 12:09 gpasero Resolution open => fixed
2017-03-06 10:43 gpasero Note Added: 0004065
2017-04-05 10:21 poughov Status resolved => closed
+Issue History