MantisBT

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0001365Orfeo Toolbox (OTB)Generalpublic2017-02-22 20:492017-03-06 10:43
Reportervfdev-5 
Assigned Togpasero 
PrioritynormalSeveritymajorReproducibilityalways
StatusresolvedResolutionfixed 
Platformx64OSMacOSXOS VersionEl Captain 10.11
Summary0001365: Python wrapper OTB-5.8.0: @rpath problem on MacOSX El Captain 10.11
Description
After environment configuration : PATH, OTB_OTB_APPLICATION_PATH. Import of otbApplication in Python ends up with :

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/otbApplication.py", line 28, in <module>
    _otbApplication = swig_import_helper()
  File "/Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/otbApplication.py", line 24, in swig_import_helper
    _mod = imp.load_module('_otbApplication', fp, pathname, description)
ImportError: dlopen(/Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/_otbApplication.so, 2): Library not loaded: @rpath/libOTBApplicationEngine-5.10.1.dylib
  Referenced from: /Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/_otbApplication.so
  Reason: image not found
```

Reason is problably in with newer MacOSX policies. Maybe, `-rpath` option need to be specified during compilation. See http://stackoverflow.com/questions/14656657/linking-a-dynamic-library-libjvm-dylib-in-mac-os-x-rpath-issue#14657625 [^]
TagsNo tags attached.
Attached Files

- Relationships

-  Notes
(0004045)
rashadkm (developer)
2017-02-24 15:08
edited on: 2017-02-24 15:13

you need to set pythonpath not the path.

I think there is an otbenv.profile in the package.

rpath is well set on compilation and on installation. Yes in both stages we take advantage of rpath!

i think you are missing only PYTHONPATH here.

(0004046)
vfdev-5 (reporter)
2017-02-24 21:35
edited on: 2017-02-24 21:36

Here is the log with otbenv.profile sourced :

```
USER:Documents user$ cd OTB-5.10.0-Darwin64/
USER:OTB-5.10.0-Darwin64 user$ ls
LICENSE Monteverdi.app bin otbenv.profile
Mapla.app README lib share
USER:OTB-5.10.0-Darwin64 user$ source otbenv.profile
USER:OTB-5.10.0-Darwin64 user$ python
Python 2.7.13 (default, Dec 17 2016, 23:03:43)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import otbApplication
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/otbApplication.py", line 28, in <module>
    _otbApplication = swig_import_helper()
  File "/Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/otbApplication.py", line 24, in swig_import_helper
    _mod = imp.load_module('_otbApplication', fp, pathname, description)
ImportError: dlopen(/Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/_otbApplication.so, 2): Library not loaded: @rpath/libOTBApplicationEngine-5.10.1.dylib
  Referenced from: /Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/_otbApplication.so
  Reason: image not found
>>>
```

Same in 5.8.0 :

```
USER:OTB-5.8.0-Darwin64 user$ ls
LICENSE Monteverdi.app bin otbenv.profile
Mapla.app README lib share
USER:OTB-5.8.0-Darwin64 user$ source otbenv.profile
USER:OTB-5.8.0-Darwin64 user$ python
Python 2.7.13 (default, Dec 17 2016, 23:03:43)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import otbApplication
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/USER/Documents/OTB-5.8.0-Darwin64/lib/python/otbApplication.py", line 28, in <module>
    _otbApplication = swig_import_helper()
  File "/Users/USER/Documents/OTB-5.8.0-Darwin64/lib/python/otbApplication.py", line 24, in swig_import_helper
    _mod = imp.load_module('_otbApplication', fp, pathname, description)
ImportError: dlopen(/Users/USER/Documents/OTB-5.8.0-Darwin64/lib/python/_otbApplication.so, 2): Library not loaded: @rpath/libOTBApplicationEngine-5.8.1.dylib
  Referenced from: /Users/USER/Documents/OTB-5.8.0-Darwin64/lib/python/_otbApplication.so
  Reason: image not found
>>>
```

(0004050)
gpasero (administrator)
2017-02-28 14:36

I can't reproduce the bug on MacOS X 10.10 (the platform used to build packages).

There is maybe something with @rpath usage, but I would need a more recent platform to investigate.
(0004051)
gpasero (administrator)
2017-02-28 14:51

As a simple test, could you try to :
* move the file /Users/USER/Documents/OTB-5.10.0-Darwin64/lib/python/_otbApplication.so to its parent directory
* add this parent directory (/Users/.../OTB-5.10.0-Darwin64/lib) to PYTHONPATH in otbenv.profile
* try again to import otbApplication from python
(0004052)
grizonnetm (administrator)
2017-02-28 14:54

Could it be related to the new security protection introduced in El Capitan?

https://en.wikipedia.org/wiki/System_Integrity_Protection [^]

@vfdev-5 can you try to disable temporarily the system protection and try again otb binaries to see if it's related?

http://osxdaily.com/2015/10/05/disable-rootless-system-integrity-protection-mac-os-x/ [^]

It cuold help us to narrow down the problem.

Thanks in advance.
(0004053)
vfdev-5 (reporter)
2017-02-28 19:39
edited on: 2017-02-28 20:21

Yes, it is related to SIP.

Unfortunately, I can not try to disable this. The problem is related with @rpath setting.

For example, the following commands help me to overcome it. Idea is to remplace @rpath by relative location with `install_name_tool`:

```
otool -L _otbApplication.so | grep -ohe "@rpath/\(\S\)*" | grep -ohe "lib\(\S\)*" | while read i; do install_name_tool -change @rpath/$i $PWD/lib/$i _otbApplication.so; done
```
and for those who use Python from brew and not system one:
```
install_name_tool -change /System/Library/Frameworks/Python.framework/Versions/2.7/Python /usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/Python _otbApplication.so
```

https://gist.github.com/vfdev-5/7d958b381e3795d96e9b0a6d7609f17b [^]

(0004054)
linkalis (reporter)
2017-03-01 07:02

I am experiencing the same issue on Mac OS Sierra version 10.12.3. I was able to disable my System Integrity Protection so that when I run `csrutil status`, I now see "System Integrity Protection status: disabled" on my Mac.

I then deleted my previous extracted OTB-5.10.0-Darwin64 folder, and re-extracted a new version of it from the OTB-5.10.0-Darwin64.run file. Then, starting from (what I hope amounts to) a "fresh" install, I'm still seeing the error:

```
ImportError Traceback (most recent call last)
<ipython-input-1-c224f4bb32fe> in <module>()
      3
      4 import numpy
----> 5 import otbApplication

/Users/USER/OrfeoToolbox/OTB-5.10.0-Darwin64/lib/python/otbApplication.py in <module>()
     26 fp.close()
     27 return _mod
---> 28 _otbApplication = swig_import_helper()
     29 del swig_import_helper
     30 else:

/Users/USER/OrfeoToolbox/OTB-5.10.0-Darwin64/lib/python/otbApplication.py in swig_import_helper()
     22 if fp is not None:
     23 try:
---> 24 _mod = imp.load_module('_otbApplication', fp, pathname, description)
     25 finally:
     26 fp.close()

ImportError: dlopen(/Users/USER/OrfeoToolbox/OTB-5.10.0-Darwin64/lib/python/_otbApplication.so, 2): Library not loaded: @rpath/libOTBApplicationEngine-5.10.1.dylib
  Referenced from: /Users/USER/OrfeoToolbox/OTB-5.10.0-Darwin64/lib/python/_otbApplication.so
  Reason: image not found
```

If it makes a difference, I'm trying to run this via an Anaconda Python 2.7 virtual environment. (I installed the Anaconda 3.6 package as my base version, and have created a separate virtual environment with Python 2.7 that I'm hoping to use for Orfeo Toolbox.) I have a file called orfeo_toolbox.pth located in my virtual environment's python2.7/site-packages folder that points to the /Users/USER/OrfeoToolbox/OTB-5.10.0-Darwin64/lib/python folder of my Orfeo install. The path seems to be detected just fine when I try to import otbApplication, because it's able to detect the otbApplication.py module. It just errors out on line 24 of the module when it tries to import the .dylib files.

I have not modified anything in the otbenv.profile file.
(0004055)
gpasero (administrator)
2017-03-01 17:55

I found the root of this issue and I could reproduce it on the platform used to build the package (MacOS X 10.10).

During package deployment, the LC_RPATH load command is added to all the executables and libraries : except on _otbApplication.so
This LC_RPATH contains the path to package lib's folder.

(0004057)
gpasero (administrator)
2017-03-02 16:34

I pushed a fix : https://git.orfeo-toolbox.org/otb.git/commit/442f70c431f78596f42a39bc40925c5f7cabcea9 [^]

The library _otbApplication.so was missing a LC_RPATH load command.
(0004064)
gpasero (administrator)
2017-03-03 12:09

Fix working fine.
(0004065)
gpasero (administrator)
2017-03-06 10:43

My thanks to the users who reported the issue. If you want to test MacOS X packages after the fix, you can download latest nightly builds at : https://www.orfeo-toolbox.org/packages/nightly/latest/ [^]

- Issue History
Date Modified Username Field Change
2017-02-22 20:49 vfdev-5 New Issue
2017-02-24 15:08 rashadkm Note Added: 0004045
2017-02-24 15:13 rashadkm Note Edited: 0004045 View Revisions
2017-02-24 21:35 vfdev-5 Note Added: 0004046
2017-02-24 21:36 vfdev-5 Note Edited: 0004046 View Revisions
2017-02-28 14:36 gpasero Note Added: 0004050
2017-02-28 14:36 gpasero Assigned To => gpasero
2017-02-28 14:36 gpasero Status new => acknowledged
2017-02-28 14:51 gpasero Note Added: 0004051
2017-02-28 14:54 grizonnetm Note Added: 0004052
2017-02-28 19:39 vfdev-5 Note Added: 0004053
2017-02-28 20:21 vfdev-5 Note Edited: 0004053 View Revisions
2017-03-01 07:02 linkalis Note Added: 0004054
2017-03-01 17:55 gpasero Note Added: 0004055
2017-03-01 17:55 gpasero Status acknowledged => confirmed
2017-03-02 16:34 gpasero Note Added: 0004057
2017-03-02 16:34 gpasero Status confirmed => assigned
2017-03-03 12:09 gpasero Note Added: 0004064
2017-03-03 12:09 gpasero Status assigned => resolved
2017-03-03 12:09 gpasero Resolution open => fixed
2017-03-06 10:43 gpasero Note Added: 0004065


Copyright © 2000 - 2017 MantisBT Team
Powered by Mantis Bugtracker