Variational Algorithms - Is there a way to avoid discontinuities in optimal variational parameters?

Question

Often algorithms utilize an ansatz $V(a)$ and rely on a classical optimization scheme over a Hamiltonian loss function $L(V(a))$ in order to find the optimal parameters $a$.

Due to many factors such as periodicity of $V(a)$, optimization path and non-convexity of $L$ (others maybe?), two problems with very similar input can result in very different optimal outputs i.e. $\forall~\delta>0 : |x_i - x_j| \leq \delta,~\exists~\epsilon>0$ such that $ | a^{\text{opt}}_i - a^{\text{opt}}_j| > \epsilon$.

I am using non-gradient optimization schemes. I have tried methods such as use constant initial parameters $a^0$, transformations e.g. $\sin(a^{\text{opt}})$, but nothing worked.

Is there a known way to avoid such discontinuities? Maybe e.g. gradient optimization schemes?

EDIT: corrected discontinuity definition

Answer 1

Let $C(\theta)$ be an arbitrary quantum circuit parametrized by $\theta \in \mathbb{R}^n$. And let $L(C(\theta))$ be a continuous non-convex objective function we would like to optimize. Given that the pair of initial parameters $\theta_1$ and $\theta_2$ are sufficiently close to a local minimum at $\theta^*$, then any deterministic gradient descent method with a sufficiently small step size will result in convergence to that local minimum for both initial $\theta_1$ and $\theta_2$. Both initial parameters don't even have to be close to each other. The only requirement for $\theta_1$ and $\theta_2$ is being in an $\epsilon$-neighborhood of the local minimum at $\theta^*$.

In practice, the obtained gradients will be noisy because of circuit sampling. So, there is a probability that even if both $\theta_1$ and $\theta_2$ are very close to a local minimum, the final estimates of optimal parameters will be different. However, if you reduce the variance of the gradients, then we will get the convergence to the same local minimum.

A nice example of a quantum natural gradient descent can be found on the Pennylane website. They provide illustrations, mathematical formulation and code.