TL;DR - Quantum neural networks use a few qubits, gates and simple quantum circuits. Regular neural networks use real numbers (composed of many bits), nonlinear functions on real numbers and very complicated underlying classical circuits, Is there some mathematical way/intution to describe the forward pass of the two different types of neural networks using a similar structure to be able to have an apples-to-apples picture of the two?
I would like to understand how quantum neural network circuits and regular neural networks can be compared, and I want to focus on just the forward pass.
In a typical quantum neural network, we have a quantum state as an input (usually a computational basis state) and then pass the state through a bunch of (parametrized) single and two-qubit linear gates, and then measure the expectation value of the state that comes out, which is a real number between 0 and 1. Typically the parameters of first few gates are decided by the input feature vector if we are doing angle encoding (and others down the line if we are implementing data-reuploading). The whole thing is basically a fairly simple quantum circuit.
In a regular neural network, we start off with a feature vector, typically composed of (approximations of) real numbers. Already there is a difference since each feature (real number) is described by a whole string of bits (compared to the quantum version where start of with separate individual qubits). Then we multiply these full feature vectors with matrices, and perform a nonlinear transformation on the outputs of these multiplications (which we can repeat this whole process several times) to finally get outputs that are also real numbers. The whole thing is a set of functions, which if we try to put into the classical circuit language, becomes extremely complicated (and, as a side note, presumably not so useful in helping us understand what is going on).
These two types of neural networks seem vastly different - especially the fact that we encode and manipulate so many bits of information in the regular neural network which has a very complicated underlying circuit, vs individual (although entangled) qubits in the quantum neural network which has a simple underlying circuit. I understand that quantum circuits explore an exponentially large space with just a few qubits, so I see why we think even small ones can be powerful, but I'm still having trouble painting a clear picture on how to compare them with regular neural networks. Is there some mathematical way (even an intuition I think would be useful) to describe the forward pass of the two different types of neural networks using a similar structure (even if it's a bit more abstract) to see how/if we can then start doing apples-to-apples comparisons between the two?