1
$\begingroup$

I'm a college student with a slight interest in quantum mechanics. I think I have a decent understanding of the Copenhagen and Many Worlds interpretations of quantum mechanics, and was considering how this could be used to improve machine learning efficiency. I want to check my understanding of quantum mechanics/computing using a design I came up with for a neural network training algorithm.

The following is a graphical representation of my algorithm. To read the diagram, follow the colored circles. The arrows show the direction in which data flows, not sequential steps in the program. The sequential steps are represented by the colored circles. Note that all state in this system would be finite.

Design for a quantum machine learning algorithm

  1. The user pre-configures their training data into the system. This consists of network input and expected network output pairs.
  2. The user pre-configures the cost threshold, a guess for the lowest accumulated cost value.
  3. The algorithm starts the iteration over training data pairs. The network input is fed into the neural network, along with the weights which are represented in qbits. This produces a network output, which is also represented in qbits. (Each superposition of the network output should be entangled with a particular superposition of the weights.) A cost function then computes a cost (represented in qbits) based on the expected network output and the network output. An accumulator accumulates these costs over each iteration.
  4. Once the iteration is finished, we compare the accumulated cost with the cost threshold. If it is less than the cost threshold, we display the weights that are entangled with the accumulated cost to the outside world. There may be multiple branches that pass the threshold, but it doesn't matter which one we output. From the outside world's perspective, the machine should have collapsed to a single set of weights that produce an accumulated cost that is less than the cost threshold. If nothing is displayed, it means no branch passed the cost threshold, so the user should start from step 2 with a higher cost threshold.
  5. Repeat steps 2, 3, 4 until satisfied with the displayed weights and cost threshold.

My idea is that by setting up the system in this way, you could reduce the problem of finding weights to a linear guess and see process (gradually increment the cost threshold one unit at a time until the machine stops displaying an output, at that point you should have found the optimal configuration of weights according to your cost function). Would this work in practice, and are there any glaring flaws with my design or understanding of quantum mechanics?

$\endgroup$
12
  • $\begingroup$ Welcome to the community! While the question is certainly detailed, I think there may not be enough math to effectively convey key differences in the architecture. I suggest comparing this approach to existing ML architectures and detail more specifically how/why a qc is being used. $\endgroup$
    – C. Kang
    Jul 10 '20 at 15:13
  • $\begingroup$ @C.Kang Yeah, I realize this isn't very formalized in terms of how quantum problems are usually posed. I was more wondering if my conceptual understanding of how entanglement and quantum branches / decoherence behave was accurate, but I can see how this could be insufficient to really answer that. I am not very familiar with the math, which is why I wrote this on a conceptual level. I might have to dig into the math and something called "quantum annealing". As far as measuring the spins of qbits for computation, I assume one would use the same plane of measurement everywhere in the machine. $\endgroup$ Jul 10 '20 at 16:21
  • $\begingroup$ I will try to add additional information when I find the time over the next day or two. $\endgroup$ Jul 10 '20 at 16:26
  • $\begingroup$ I'd recommend by starting with an understanding of qubits / quantum algorithms. QC is really all math $\endgroup$
    – C. Kang
    Jul 10 '20 at 16:52
  • 2
    $\begingroup$ @JerryFielder if the input of the NN is a quantum state, then you cannot simply apply the map corresponding to the NN to the vector representing the quantum state. NNs are inherently nonlinear functions, by design. Quantum mechanics only allows for linear mappings between input and output state vectors. Any nonlinearity requires probabilistic schemes, and if you go this direction you need to be very careful about the actual probabilities at all stages. This is not an easy problem to tackle. $\endgroup$
    – glS
    Jul 14 '20 at 6:41
1
$\begingroup$

A few things I would consider:

  • If you have a superposition of the data, weights, etc. I suspect (but am not entirely confident) that you may find a fully quantum neural network reduces to something very similar to a Grover's search over the network weights. An example of a similar reduction exists in quantum genetic algorithms (see this paper and this one) where the entire population can be encoded into a uniform superposition and, given some fitness function (i.e. the oracle), we can find the individual that maximizes the fitness function by amplitude amplification. This obviates the need for the traditional operators of genetic algorithms – crossover, mutation, etc.
  • The weights you "entangle with the observer" (I'd urge against framing the measurement/output of an algorithm in this way), while appropriate for your quantum neural network, may not be transferrable to a classical neural network (given the lack of entanglement available to a classical neural network) or efficiently encoded into another quantum neural network (I'm not sure if this is what you hoped to do).
  • Using the epistemological/philosophical interpretations of quantum mechanics to formulate machine learning models may not be the best approach. Sure, one of the interpretations may particularly resonate and aid in the comprehension of what's going on therein, but they are hardly design principles and – to my knowledge – interpretations by the very fact that we find it difficult (or impossible) to distinguish which one may be "correct" through experiment, though I'm also not a physicist so caveat emptor.
  • I'm not seeing any mention of back propagation or any of the other classic optimization algorithms used in neural networks, though it appears you're using a parameterized signum as an activation. But how does this algorithm actually learn? You somewhat address it in 4, but the approach suggested seems quite crude. You mention increasing the cost threshold – sure, but by how much?

Now, all that said, the good news is that many quantum neural network architectures already exist that can be used to draw inspiration. Here are a few examples:

These are just a handful of neural network designs (they all have arXiv papers that are free to access, which I should have linked to, but I pulled these from a working bibliography). They should provide a general idea of the types of QNNs that are currently being developed and implemented. Hope this helps!

$\endgroup$
5
  • $\begingroup$ It's been a while but I remember now. I was basing this off the many worlds interpretation, so maybe total nonsense when it comes to real practice. The idea was that the interior of the system (call it S) is quantumly isolated from the observer, except through the display. This isolation allows S to exist in a superposition so that each permutation of possible weights has a branch. A cost threshold is set before a "training" cycle, and it tells any branch resulting in a cost lower than itself to reveal itself to the observer/outside world (by displaying the weights of that branch). $\endgroup$ Aug 8 at 4:09
  • $\begingroup$ So you would gradually lower the cost threshold until the system takes an unusually long time to return an output, at which point you can assume it has outputted the optimal weights. $\endgroup$ Aug 8 at 4:11
  • $\begingroup$ If you presented this to me as, say, a proposal for some method, the biggest critique I would have to be with your training approach. Specifically, 1. “lowering the cost threshold” — how? And under what schedule? And 2. you assume it has converged to the optimal weights after “an unusually long time.” How long? Can you formulate the analytical bounds on the run time of the algorithm? Will it at least do it in polynomial time? $\endgroup$
    – Greenstick
    Aug 10 at 2:58
  • $\begingroup$ I ask these questions because your training method doesn’t seem aware of modern optimization approaches. For example, in classical neural nets (and some quantum ones), back propagation is used, which computes the derivative of the weights with respect to the error throughout the network layers using the chain rule. In lay terms, that means we can estimate how much to change the weights by (to get closer to the optimum) — imagine an error landscape; we’re essentially making the algorithm aware of both the direction to move and by how much. But even this form of SGD is the most naïve approach. $\endgroup$
    – Greenstick
    Aug 10 at 3:06
  • $\begingroup$ Also, SGD (in case your not familiar) is stochastic gradient descent — had to abbreviate cause of length. Anyway, I hope this helps. If you’re interested in classical optimization of neural networks (a good starting point), this article gives a nice overview of the challenges of learning neural network weights well (there are many!) and some approaches to get around them. $\endgroup$
    – Greenstick
    Aug 10 at 3:12
0
$\begingroup$

On paper I think this a cool idea. Though the terms you are using like cost threshold, cost function, and weights will all have to be transformed. Classically this a well thought out idea and then to ask "If done on classical can I just tweak a few things and make them quantum?" is an awesome idea. The terms you used must be extended to the quantum system you are using. Lets say your neural network outputs unique binary strings which represent each of the qubits state. Ok, from here now we must figure out which sort of quantum algorithm to use on our qubits to find a 'quantum' cost, then we keep this information and for every box that takes in qubits you must find a quantum algorithm to emulate the problem solving done classically. Luckily, quantum optimization is a blossoming field and there have been very cool papers recently, I suggest a google on search on quantum optimization and quantum machine learning. One last thought: will this system be more efficient than the classical system and produce better results?

Word of advice that was once passed on to me on the subject of quantum computing: The quantum computer breaks all intuition that classical computers have given us, it is best to leave this classical intuition at the door and observe these quantum algorithms like a baby.

Essentially it will be easier to start with a quantum system and translate it's power over to classical computing. It is much harder to start with a classical idea and translate it over to quantum, not that it cannot be done, but rather in terms of optimizing your own time and resources to go from classical to quantum is like finding a needle in a haystack.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.