TL,DR: Engineering and physics arguments have already been made. I add a historical perspective: I argue that the field of quantum computation is really only a bit more than two decades old and that it took us more than three decades to build something like the MU5.
Since you mention the timeline, let's have a closer look:
The beginnings
First of all, the mere possibility of something like a quantum computer was voiced by Richard Feynman in the west (1959 or 1981 if you wish) and Yuri Manin in the east (1980). But that's just having an idea. No implementation starts.
When did similar things happen with classical computing? Well, a very long time ago. Charles Babbage for instance already wanted to build computing machines in the early 19th century and he already had ideas. Pascal, Leibniz, they all had ideas. Babbage's analytical machine of 1837 which was never built due to funding and engineering challenges (by the way, the precursor of the analytical machine was built with Lego) is definitely the most recent first idea that is already way ahead of what Feynman and Manin proposed for quantum computing, because it proposes a concrete implementation.
The '70s don't see anything related to a quantum computer. Some codes are invented, some theoretical groundwork is done (how much information can be stored?), which is necessary for qc, but it's not really pursuing the idea of the quantum computer.
Codes and communication-related ideas are to quantum computation what telephones and telegraph wires are to classical computing: an important precursor, but not a computer. As you know, Morse codes and telegraphs are technologies of the 19th century and more difficult codes for noisy channels were also studied. The mathematical groundwork (in terms of no-go-theorems and the like) was done in 1948 by Shannon.
Anyway, it can be argued that punch card computing was developed in 1804 for weaving, but I don't want to claim that this was really the beginning of the classical computation.
Universal (quantum) computers
So when did computation start? I'm going to argue that you need a number of things to get research for universal computing off the ground; before that, the number of people and money invested there will be limited.
- You need the notion of a universal computer and a theoretical model of what to achieve.
- You need an architecture of how to implement a universal computer - on a theoretical level.
- You need a real-life system where you could implement it.
When do we get those in quantum computation?
- Deutsch describes the universal quantum computer in 1985 (33 years ago).
- Circuit models and gates are developed around the same time.
- The first complete model of how to put everything together was proposed by Cirac and Zoller in 1994 (merely 24 years ago).
All the other advances in quantum computation before or during that time were limited to cryptography, quantum systems in general or other general theory.
What about classical computation?
- We have Turing's work on Turing machines (1936) or Church's work (same time frame).
- Modern architectures rely on von Neumann's model (1945); other architectures exist.
- As a model, the digital circuit model was designed in 1937 by Shannon.
So, in 1994 we are in a comparable state to 1937:
- There are a few people doing the theoretical groundwork, and the groundwork has now been done.
- There are a fair number of people doing engineering work on foundational issues not directly related but very helpful for building a (quantum) computer.
- And the field is generally not that big and well-funded.
- But: from that date, funding and people start pouring into the field.
The field is taking off
For classical computing, this is illustrated by the amount of different "first computer systems" in the Wikipedia timeline. There were several research groups at least in Germany, England, and the United States in several locations (e.g. Manchester and Bletchley Park in the UK, to name just a few). War-time money was diverted to computing because it was necessary for e.g. the development of the nuclear bomb (see accounts at Los Alamos).
For quantum computation, see e.g. this comment:
The field of QIS began an explosive growth in the early to mid-1990s as a consequence of several simultaneous stimuli: Peter Shor demonstrated that a quantum computer could factor very large numbers super-efficiently. The semiconductor industry realized that the improvement of computers according to Moore’s law would all too soon reach the quantum limit, requiring radical changes in technology. Developments in the physical sciences produced trapped atomic ions, advanced optical cavities, quantum dots, and many other advances that made it possible to contemplate the construction of workable quantum logic devices. Furthermore, the need for secure communications drove the investigations of quantum communication schemes that would be tamper proof.
All in all, from the time, that the theoretical groundwork of modern computers had been laid to the time that the first computers are available (Zuse 1941, Manchester 1948, to name just two) it took about a decade. Similarly, it took about a decade for the first systems doing some sort of universally programmable calculation with quantum systems. Granted, their capabilities are lower than the first Manchester computers, but still.
Twenty years later, we are slowly seeing explosive growth in technology and a lot of firms get involved. We also see the advent of new technologies like the transistor (first discovered in 1947).
Similarly, 20 years after the beginning of quantum computation we see the serious entrance of private companies into the field, with Google, IBM, Intel, and many others. When I was at my first conference in 2012, their involvement was still academic, today, it is strategic. Similarly, we saw a proposition of a wealth of different quantum computing systems during the 2000s such as superconducting qubits, which form the basis of the most advanced chips from the three companies mentioned above. In 2012, nobody could claim to have a somewhat reliable system with more than a couple of physical qubits. Today, only six years later, IBM lets you play with their very reliable 16 qubits (5 if you really only want to play around) and Google claims to test a 72 qubit system as we speak.
Yes, we have still some way to go to have a reliable large-scale quantum computer with error-correction capabilities, and the computers we currently have are weaker than the classical computers we had in the '60s, but I (as others explain in other answers) believe this is due to the unique engineering challenges.
There is a small chance that it's due to physical limitations we have no idea about but if it is, given current progress, we should know in a couple of years at the latest.
What's my point here?
- I argued that the reason that we don't see an MU5 quantum computer yet is also due to the fact that the field is just not that old, yet, and hasn't really achieved that much attention until recently.
- I argue that from a present-day perspective, it seemed that classical computers became very good very quickly, but that this neglects decades of prior work where development and growth didn't seem as fast.
- I argue that if you believe (as almost everybody in the field does) that the initial engineering problems faced by quantum computers are harder than those faced by classical computers, then you see a very much comparable research and innovation trajectory to one of the classical computers. Of course, they are somewhat different, but the basic ideas of how it goes are similar.