I hope this is the right place to ask this kind of question. I'm using the Aer simulator in Qiskit to analyze the state vector of some circuits. My program is very parallelizable, but when I try to run these qiskit calculations in parallel (using the multiprocessing library in python), in certain situations the call to Qiskit to simulate the circuit hangs indefinitely. For instance, I may have something like this:
from multiprocessing import Pool
def parallel(circ2):
simulator2 = Aer.get_backend('aer_simulator')
result2 = simulator.run(circ2).result()
print(result2)
circ1 = ... #some kind of circuit
simulator1 = Aer.get_backend('aer_simulator')
result1 = simulator1.run(circ1).result()
print(result1)
circs_arr = [...] #some list of different circuits
with Pool(5) as p:
p.map(parallel,circs_arr)
The above code does not work. The parallelized call to run the circuit (within the function) hangs indefinitely. However, if I remove the first run of the simulator (that produces result1), then everything is fine and works as expected. Does anyone know how I can fix this? I've read that the problem might have something to do with qiskit Aer utilizing multiprocessing for its own purposes, so using running Aer with multiprocessing can cause a conflict. However, my attempts to turn off qiskit's internal multiprocessing have not worked, or have not fixed my problem.