Quantum simulation is advancing and I'm wondering if now or in the future there is a point where its more cost effective and efficient to run general purpose parallel algorithms (e.g. with a mix of operations, loops, conditional flows and highly parallel tasks etc) on an hardware/GPU/analogue accelerated quantum simulator, or will it be always better on CPUs?
We are not at the point where quantum computers are outperforming classical computers in any practical way so certainly not now. As for the future, there already exist numerous algorithms which have been shown to theoretically outperform classical computers so once the hardware catches up we should be in a good place to take advantage of it.
Regarding the cost effectiveness, there was a recent assessment which looked at the operating costs of running Google's quantum supremacy calculation on a QC and a classical computer and they estimated the following:
For the classical computer, we obtain: $$ 60 h * 13 000 kW * $0.1/kWh = $78,000. $$ For the quantum computer, we obtain: $$ 0.056 h * 25 kW * $0.1/kWh = $0.14. $$
Following up with your comment, while simulating a quantum computer you need to hold every complex amplitude in memory. Since this has an exponential scaling, it isn't possible to do this efficiently on a classical computer. For example, if you have $N$ qubits then you need to store in memory $2^N$ complex amplitudes which quickly becomes infeasible for even our most powerful supercomputers. Feynman discussed this early on and you read about it in section 3: simulating probability. Given this type of scaling, you won't be able to access the power of a quantum computing algorithm using GPU's over CPU's, but perhaps you can get some speedups.
"I was actually looking for something more related to simulation of quantum computers. If simulation is efficient enough to be more efficient that classical computing, then we can start writing quantum computing algorithms to get direct benefits now, that's the idea i'm thinking of."
Since classical simulation of quantum computers on non-quantum hardware, falls under the category of classical computing, it is impossible for it to become "more efficient" than classical computing, because it's an example of classical computing, meaning that classical computing will be able to offer something at least as efficient (in the worst case the classical algorithm can be simply the simulation of the quantum computer on a classical computer, which is what you started with in this comparison).
"The current response does not actually answer the question. I want to understand the efficiency of parallel algorithms via quantum simulation on accelerated hardware Vs CPU, not on actual quantum computers."
You have given GPUs as an example of "accelerated hardware", and I suppose you're aware of other types of "accelerated non-CPU hardware" such as FPGAs and ASICs. All of those examples of "accelerated hardware" are examples of classical computers, and yes they do vastly outperform ordinary CPUs in many cases, even for the simulation of the quantum mechanics going on in quantum computers. However there is not one example where simulating a quantum computer on a classical device (whether it be a CPU or "accelerated hardware" such as GPUs) leads us to a more efficient way to solve a problem than just solving the problem directly on a classical device.