2
$\begingroup$

Qiskit Aqua raises the following exception:

ImportError: dlopen: cannot load any more object with static TLS

I don't understand what might be causing this. I currently have a Ubuntu 14.04 Linux distribution and am using Anaconda Navigator, which I have downloaded with Conda. See code and full error log below.


Code:

from qiskit import *
from numpy import *
import qiskit as q
from math import pi
import numpy as np
from numpy.fft import fft
import scipy.fftpack
import matplotlib.pyplot as plt
import matplotlib.pyplot as plt1
import matplotlib.pyplot as plt2
import matplotlib.pyplot as circuit
from qiskit.quantum_info import Statevector
from qiskit.quantum_info import DensityMatrix
from qiskit.visualization import plot_bloch_multivector, plot_histogram
from qiskit.visualization import plot_state_city
from typing import List, Union
from qiskit.quantum_info import Pauli
import time
from itertools import combinations
from qiskit.compiler import assemble
from qiskit.test.mock import FakeVigo
from qiskit.circuit import Parameter
from qiskit import QuantumRegister, ClassicalRegister, QuantumCircuit
from qiskit.tools.visualization import circuit_drawer
from matplotlib import style 
from qiskit import execute, BasicAer
from qiskit.compiler import transpile
from qiskit.quantum_info.operators import Operator, Pauli
from qiskit.quantum_info import process_fidelity
from qiskit.extensions import RXGate, XGate, CXGate
from qiskit import QuantumCircuit
from qiskit.circuit import ParameterExpression
from typing import Union, Set, List
import numpy as np

from qiskit.circuit import ParameterExpression
from qiskit.aqua.operators import PrimitiveOp



%matplotlib inline
#%matplotlib tk


#style.use("classic")
COLOR = 'white'
COLOR1 = 'black'
plt.rcParams['text.color'] = COLOR1
plt.rcParams['axes.labelcolor'] = COLOR
plt.rcParams['xtick.color'] = COLOR
plt.rcParams['ytick.color'] = COLOR

plt1.rcParams['text.color'] = COLOR1
plt1.rcParams['axes.labelcolor'] = COLOR
plt1.rcParams['xtick.color'] = COLOR
plt1.rcParams['ytick.color'] = COLOR

plt2.rcParams['text.color'] = COLOR1
plt2.rcParams['axes.labelcolor'] = COLOR
plt2.rcParams['xtick.color'] = COLOR
plt2.rcParams['ytick.color'] = COLOR


#style.use("seaborn-whitegrid")



plt.rc('legend',fontsize=20) # using a size in points
plt1.rc('legend',fontsize=20) # using a size in points
plt2.rc('legend',fontsize=20) # using a size in points

#Rotation Parameters:
#N = .5

Error:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-4-f30f52024441> in <module>
     35 
     36 from qiskit.circuit import ParameterExpression
---> 37 from qiskit.aqua.operators import PrimitiveOp
     38 
     39 

~/anaconda3/lib/python3.8/site-packages/qiskit/aqua/__init__.py in <module>
     74 from .missing_optional_library_error import MissingOptionalLibraryError
     75 from .aqua_globals import aqua_globals
---> 76 from .quantum_instance import QuantumInstance
     77 from .algorithms import QuantumAlgorithm
     78 from ._logging import (QiskitLogDomains,

~/anaconda3/lib/python3.8/site-packages/qiskit/aqua/quantum_instance.py in <module>
     32 
     33 from .aqua_error import AquaError
---> 34 from .utils.backend_utils import (is_ibmq_provider,
     35                                   is_statevector_backend,
     36                                   is_simulator_backend,

~/anaconda3/lib/python3.8/site-packages/qiskit/aqua/utils/__init__.py in <module>
     58 from .subsystem import get_subsystem_density_matrix, get_subsystems_counts
     59 from .entangler_map import get_entangler_map, validate_entangler_map
---> 60 from .dataset_helper import (get_feature_dimension, get_num_classes,
     61                              split_dataset_to_data_and_labels,
     62                              map_label_to_class_name, reduce_dim_to_via_pca)

~/anaconda3/lib/python3.8/site-packages/qiskit/aqua/utils/dataset_helper.py in <module>
     16 from copy import deepcopy
     17 import numpy as np
---> 18 from sklearn.decomposition import PCA
     19 
     20 

~/anaconda3/lib/python3.8/site-packages/sklearn/__init__.py in <module>
     79     from . import __check_build  # noqa: F401
     80     from .base import clone
---> 81     from .utils._show_versions import show_versions
     82 
     83     __all__ = ['calibration', 'cluster', 'covariance', 'cross_decomposition',

~/anaconda3/lib/python3.8/site-packages/sklearn/utils/_show_versions.py in <module>
     10 import importlib
     11 
---> 12 from ._openmp_helpers import _openmp_parallelism_enabled
     13 
     14 

ImportError: dlopen: cannot load any more object with static TLS
$\endgroup$
3
  • 1
    $\begingroup$ This issue seems to be related: github.com/scikit-learn/scikit-learn/issues/14485 maybe it helps $\endgroup$
    – Cryoris
    Jan 20 at 6:47
  • $\begingroup$ Other related issues: github.com/pytorch/pytorch/issues/2575 and github.com/Kaggle/docker-python/issues/206 . Have you tried to move up the qiskit.aqua import? As a side note, a lot of your imports are either discouraged, duplicated or at least unordered. Cleaning a little bit those might help as well. $\endgroup$ Jan 20 at 7:43
  • $\begingroup$ @AdrienSuau I've tried to move up the qiskit.aqua import but it didn't help. I will be cleaning up my code, although I do have another code with all these imports (without the qiskit.aqua one) that works, so hopefully they're not causing a problem. Thank you for your help though, I'll check the answer submitted and hopefully it'll work. $\endgroup$ Jan 22 at 4:14
1
$\begingroup$

@Adam Zalcman answer explains well why the issue happens.

According to tensorforce/issues/303:

Linux has a static limit on the number of shared libraries with TLS (Thread-Local Storage, to support C++'s __thread storage class) that can be loaded into a process. Gym itself doesn't use shared libraries, but OpenGL and Tensorflow both have several libraries. The limit is small, like 14 or 32 depending on OS version (it's the version of glibc that matters).

The issue was solved by upgrading from Ubuntu 14 to Ubuntu 16.

This could potentially solve the issue as upgrading the OS will likely also upgrade the glibc shipped with the OS. It seems like the version of glibc shipped with Ubuntu 16 increased the "static limit on the number of shared libraries with TLS".

$\endgroup$
3
  • $\begingroup$ Going to do so and hopefully things will go well. Thank you for all the help. $\endgroup$ Jan 25 at 2:19
  • $\begingroup$ Keep us updated, and if it solved your issue remember to check the answer as the accepted one. It might help others with the same issue in the future. $\endgroup$ Jan 25 at 9:07
  • 2
    $\begingroup$ Just wanted to update you that it worked! Thanks for all the help. $\endgroup$ Jan 26 at 6:54
1
$\begingroup$

Hypothesis

OpenMP uses static Thread Local Storage which requires slots in a data structure called Dynamic Thread Vector. Unfortunately, DTV is allocated based on the needs of the libraries statically linked into the executable. The needs of dynamically loaded libraries are served from a fixed surplus of slots on the first come first served basis. If a library loaded using dlopen uses static TLS but no more DTV slots are available then dlopen fails the way you see.

See also here and here.

Potential workaround

You may be able to work around the problem by forcing OpenMP to be loaded earlier, when DTV slots are still available. First, find out what OpenMP library you are using and where it is. A common OpenMP library is GOMP and the file is usually called libgomp.so etc. Next, set the LD_PRELOAD environment variable to its absolute path using

export LD_PRELOAD="/absolute/path/to/libgomp.so"

and then re-run your code.

$\endgroup$
6
  • $\begingroup$ Unfortunately, this workaround hasn't worked in my case. I'm not really sure how to go about this from here now. $\endgroup$ Jan 22 at 4:48
  • $\begingroup$ Next, I would try setting LD_PRELOAD not to a path, but bare filename (i.e. no slashes in the name). IIRC, the shared object cache treats different paths as different objects, so dlopen may fail to find the preloaded OpenMP library if it was loaded under a different name than the one used in dlopen call. Note that if this prevents the dynamic linker from finding the library then you may also need to include the directory where OpenMP library resides in the LD_LIBRARY_PATH variable. $\endgroup$ Jan 22 at 5:18
  • $\begingroup$ If the above fails, upgrading to Ubuntu 16 should help, see @Adrien Suau's answer below. $\endgroup$ Jan 24 at 3:25
  • 1
    $\begingroup$ I'm going to upgrade to Ubuntu 16 as recommended. Hopefully that will work. Thank you for all the help. $\endgroup$ Jan 25 at 2:18
  • 1
    $\begingroup$ It thankfully did fix the problem! Thank you for all the help. $\endgroup$ Jan 26 at 6:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.