How to benchmark a quantum computer?

Question

Using a simple puzzle game to benchmark quantum computers is the most clever approach I have seen so far.

The author of the aforementioned article, James, makes a nice analogy to buying a laptop ("more than just a single number when comparing") in an answer to How should different quantum computing devices be compared?.

Another answer by whurley to the same question mentions the IEEE Standard for Quantum Computing Performance Metrics & Performance Benchmarking.

In When will we know that quantum supremacy has been reached? there is an answer by Niel which includes this snippet:

But the bottom line is that, yes, "quantum supremacy" is precisely about "not being able to simulate quantum computers of certain sizes", or at least not being able to simulate certain specific processes that you can have them perform, and this benchmark depends not only on quantum technology but on the best available classical technology and the best available classical techniques. It is a blurry boundary which, if we are being serious about things, we will only be confident that we have passed a year or two after the fact. But it is an important boundary to cross.

My question is bipartite:

• What will be the primary metrics for quantum computers? (Clasical examples include processor speed, RAM & hard drive space)

• Best tools / strategies for benchmarking above metrics? (Existing or proposed)

Answer 1

This may not exactly answer your question (which I suspect is still very much an open question, and what you're likely to get as answers are opinions), but have you looked at blind quantum computation? See here for another perspective.

One way that we can describe that premise is to imagine some company claims to have developed a fabulous universal quantum computer. But it's so expensive, difficult to control etc. that only they can be trusted to run stuff on it, and it's not directly open for use by other people. How do you know it's really running as a quantum computer, and not just that, for example, they have some new classical algorithm that can simulate quantum computations better than we were expecting?

Blind quantum computation achieves this by making the quantum computer perform a computation without knowing what that computation actually is! The company who owns the computer is left blind to what it's supposed to be doing, so their cheating strategies are severely curtailed.

However, I assume that this method would not be applicable to the quantum supremacy scenario.