Error When Importing from qiskit.aqua

Question

Qiskit Aqua raises the following exception:

ImportError: dlopen: cannot load any more object with static TLS

I don't understand what might be causing this. I currently have a Ubuntu 14.04 Linux distribution and am using Anaconda Navigator, which I have downloaded with Conda. See code and full error log below.

---

Code:

from qiskit import *
from numpy import *
import qiskit as q
from math import pi
import numpy as np
from numpy.fft import fft
import scipy.fftpack
import matplotlib.pyplot as plt
import matplotlib.pyplot as plt1
import matplotlib.pyplot as plt2
import matplotlib.pyplot as circuit
from qiskit.quantum_info import Statevector
from qiskit.quantum_info import DensityMatrix
from qiskit.visualization import plot_bloch_multivector, plot_histogram
from qiskit.visualization import plot_state_city
from typing import List, Union
from qiskit.quantum_info import Pauli
import time
from itertools import combinations
from qiskit.compiler import assemble
from qiskit.test.mock import FakeVigo
from qiskit.circuit import Parameter
from qiskit import QuantumRegister, ClassicalRegister, QuantumCircuit
from qiskit.tools.visualization import circuit_drawer
from matplotlib import style 
from qiskit import execute, BasicAer
from qiskit.compiler import transpile
from qiskit.quantum_info.operators import Operator, Pauli
from qiskit.quantum_info import process_fidelity
from qiskit.extensions import RXGate, XGate, CXGate
from qiskit import QuantumCircuit
from qiskit.circuit import ParameterExpression
from typing import Union, Set, List
import numpy as np

from qiskit.circuit import ParameterExpression
from qiskit.aqua.operators import PrimitiveOp



%matplotlib inline
#%matplotlib tk


#style.use("classic")
COLOR = 'white'
COLOR1 = 'black'
plt.rcParams['text.color'] = COLOR1
plt.rcParams['axes.labelcolor'] = COLOR
plt.rcParams['xtick.color'] = COLOR
plt.rcParams['ytick.color'] = COLOR

plt1.rcParams['text.color'] = COLOR1
plt1.rcParams['axes.labelcolor'] = COLOR
plt1.rcParams['xtick.color'] = COLOR
plt1.rcParams['ytick.color'] = COLOR

plt2.rcParams['text.color'] = COLOR1
plt2.rcParams['axes.labelcolor'] = COLOR
plt2.rcParams['xtick.color'] = COLOR
plt2.rcParams['ytick.color'] = COLOR


#style.use("seaborn-whitegrid")



plt.rc('legend',fontsize=20) # using a size in points
plt1.rc('legend',fontsize=20) # using a size in points
plt2.rc('legend',fontsize=20) # using a size in points

#Rotation Parameters:
#N = .5

Error:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-4-f30f52024441> in <module>
     35 
     36 from qiskit.circuit import ParameterExpression
---> 37 from qiskit.aqua.operators import PrimitiveOp
     38 
     39 

~/anaconda3/lib/python3.8/site-packages/qiskit/aqua/__init__.py in <module>
     74 from .missing_optional_library_error import MissingOptionalLibraryError
     75 from .aqua_globals import aqua_globals
---> 76 from .quantum_instance import QuantumInstance
     77 from .algorithms import QuantumAlgorithm
     78 from ._logging import (QiskitLogDomains,

~/anaconda3/lib/python3.8/site-packages/qiskit/aqua/quantum_instance.py in <module>
     32 
     33 from .aqua_error import AquaError
---> 34 from .utils.backend_utils import (is_ibmq_provider,
     35                                   is_statevector_backend,
     36                                   is_simulator_backend,

~/anaconda3/lib/python3.8/site-packages/qiskit/aqua/utils/__init__.py in <module>
     58 from .subsystem import get_subsystem_density_matrix, get_subsystems_counts
     59 from .entangler_map import get_entangler_map, validate_entangler_map
---> 60 from .dataset_helper import (get_feature_dimension, get_num_classes,
     61                              split_dataset_to_data_and_labels,
     62                              map_label_to_class_name, reduce_dim_to_via_pca)

~/anaconda3/lib/python3.8/site-packages/qiskit/aqua/utils/dataset_helper.py in <module>
     16 from copy import deepcopy
     17 import numpy as np
---> 18 from sklearn.decomposition import PCA
     19 
     20 

~/anaconda3/lib/python3.8/site-packages/sklearn/__init__.py in <module>
     79     from . import __check_build  # noqa: F401
     80     from .base import clone
---> 81     from .utils._show_versions import show_versions
     82 
     83     __all__ = ['calibration', 'cluster', 'covariance', 'cross_decomposition',

~/anaconda3/lib/python3.8/site-packages/sklearn/utils/_show_versions.py in <module>
     10 import importlib
     11 
---> 12 from ._openmp_helpers import _openmp_parallelism_enabled
     13 
     14 

ImportError: dlopen: cannot load any more object with static TLS

Answer 1

@Adam Zalcman answer explains well why the issue happens.

According to tensorforce/issues/303:

Linux has a static limit on the number of shared libraries with TLS (Thread-Local Storage, to support C++'s __thread storage class) that can be loaded into a process. Gym itself doesn't use shared libraries, but OpenGL and Tensorflow both have several libraries. The limit is small, like 14 or 32 depending on OS version (it's the version of glibc that matters).

The issue was solved by upgrading from Ubuntu 14 to Ubuntu 16.

This could potentially solve the issue as upgrading the OS will likely also upgrade the glibc shipped with the OS. It seems like the version of glibc shipped with Ubuntu 16 increased the "static limit on the number of shared libraries with TLS".

Answer 2

Hypothesis

OpenMP uses static Thread Local Storage which requires slots in a data structure called Dynamic Thread Vector. Unfortunately, DTV is allocated based on the needs of the libraries statically linked into the executable. The needs of dynamically loaded libraries are served from a fixed surplus of slots on the first come first served basis. If a library loaded using dlopen uses static TLS but no more DTV slots are available then dlopen fails the way you see.

See also here and here.

Potential workaround

You may be able to work around the problem by forcing OpenMP to be loaded earlier, when DTV slots are still available. First, find out what OpenMP library you are using and where it is. A common OpenMP library is GOMP and the file is usually called libgomp.so etc. Next, set the LD_PRELOAD environment variable to its absolute path using

export LD_PRELOAD="/absolute/path/to/libgomp.so"

and then re-run your code.