ADDED

Much of your concerns were emphasized early on, especially after Shor's algorithm was introduced - but, the feeling of the community was that these were addressed with the proof of the Threshold Theorem which, although I'm not an expert on, notes that if an error rate per gate can be kept below a certain value then error correction wins in the long run. As indicated above quantum computing can win out in the long run by reducing the number of gates required. The error correction likewise only uses a linear(ish) number of additional qubits.

Your comment that you are "referring to fundamental physical operations and not gates" is inscrutable to me. Quantum gates are physical operations - e.g. microwave pulses on a coupler or lasers fired at ions.

Additionally I read the Google paper you linked to as throwing cold-water on being able to achieve a quantum computational advantage by running Grover's algorithm with the currently existing error correcting schemes. They did not show that Grover's algorithm scales as $O(N)$ as you seem to assert; rather, it emphasizes that the advantage of Grover's algorithm may be difficult to realize with existing hardware and error-correcting schemes. There is a legitimate criticism that much of theoretical computer science ignores constant overheads, but the assertion that "a QC needing $S$ operations takes at least $O(S^2)$ time to complete" is not established by your question or your linked papers.

I think there's some difficulty in communication and a lack of common language between physics and computer science, that make the question hard to answer. But I'll try to give some the computer science perspectives.

Initially I emphatically reject the assertion that there's a massive refutation of most quantum computation claims, because as has been mentioned many times quantum computers provide speedups by reducing the number of needed operations and not by reducing the amount of time required to perform any particular operation. Indeed, most gate operations would likely take significantly longer on a quantum computer than on a classical computer. But, there's just so many fewer operations needed on the quantum computer. I will quote Shor himself:

My favorite analogy is with transportation. Think of a quantum computer as a boat and a classical computer as a car. Suppose you want to go from New London, CT to Orient, NY. The ferry will take 80 minutes. Google Maps says the distance is 210 miles. So clearly, the ferry is averaging 157.5 miles per hour, right? No, it's taking a different path that is shorter (but that only boats can take). Similarly, Shor's algorithm is taking a different path that is shorter (but that only quantum computers can take).

Two other points for consideration:

1. In the comments the statement is provided that "the more reversible the machine, the slower the computation. Correct?" But, since the 70's/early 80's, we've known how to take any classical circuit that is irreversible, and convert the circuit to one that's reversible, at the cost of doubling the number of gates or operations. This has been referred to as Bennett's trick of uncomputation. (If the circuit uses recursion then there are some other issues, but in general this doubling is applicable). So yes, a reversible circuit will use more gates than an irreversible one, but only up to twice-as-many most of the time. If, as is claimed by Grover, a quantum computer can run in square-root-of-time, then the doubling of the circuit length to make it reversible is easily swamped out by the quadratic improvement in the number of calls to the circuit.

2. The appeal to the time-energy uncertainty principle is not misplaced! And indeed there is a theorem in quantum computing directly analogous to this principle - namely, the idea that most Hamiltonians admit no fast-forwarding. This means that for many Hamiltonians, the only way to get very accurate estimates of a Hamiltonian's energy is by actively running the simulation for longer and longer times. There are interesting exceptions to the no fast-forwarding theorem - with Shor's algorithm being the prototypical example.

^{I'd recommend the OP avoid any continued repetition of statements such as "[u]nless you can disprove my lower bound on QC runtime, it's a massive refutation of most QC claims", because the tone of such statements is a distraction from the potentially interesting points that the OP wants to make.}

I think there's some difficulty in communication and a lack of common language between physics and computer science, that make the question hard to answer. But I'll try to give some the computer science perspectives.

Initially I emphatically reject the assertion that there's a massive refutation of most quantum computation claims, because as has been mentioned many times quantum computers provide speedups by reducing the number of needed operations and not by reducing the amount of time required to perform any particular operation. Indeed, most gate operations would likely take significantly longer on a quantum computer than on a classical computer. But, there's just so many fewer operations needed on the quantum computer. I will quote Shor himself:

My favorite analogy is with transportation. Think of a quantum computer as a boat and a classical computer as a car. Suppose you want to go from New London, CT to Orient, NY. The ferry will take 80 minutes. Google Maps says the distance is 210 miles. So clearly, the ferry is averaging 157.5 miles per hour, right? No, it's taking a different path that is shorter (but that only boats can take). Similarly, Shor's algorithm is taking a different path that is shorter (but that only quantum computers can take).

Two other points for consideration:

1. In the comments the statement is provided that "the more reversible the machine, the slower the computation. Correct?" But, since the 70's/early 80's, we've known how to take any classical circuit that is irreversible, and convert the circuit to one that's reversible, at the cost of doubling the number of gates or operations. This has been referred to as Bennett's trick of uncomputation. (If the circuit uses recursion then there are some other issues, but in general this doubling is applicable). So yes, a reversible circuit will use more gates than an irreversible one, but only up to twice-as-many most of the time. If, as is claimed by Grover, a quantum computer can run in square-root-of-time, then the doubling of the circuit length to make it reversible is easily swamped out by the quadratic improvement in the number of calls to the circuit.

2. The appeal to the time-energy uncertainty principle is not misplaced! And indeed there is a theorem in quantum computing directly analogous to this principle - namely, the idea that most Hamiltonians admit no fast-forwarding. This means that for many Hamiltonians, the only way to get very accurate estimates of a Hamiltonian's energy is by actively running the simulation for longer and longer times. There are interesting exceptions to the no fast-forwarding theorem - with Shor's algorithm being the prototypical example.