Around 1960-1973 the idea was beginning to form, but the field really started spreading in the 1980s.
One of the biggest pioneers was Richard P. Feynman. He proposed a model of a quantum computer in his talk. From that talk, many other scientists pushed the field further (Toffoli created one of the first quantum gates; Shor, at Bell Labs, created one of the first quantum algorithms etc.)
The field has been changing and evolving rapidly throughout 1980-2000, and keeps evolving. But the initial "spark" was made by Richard Feynman. I don't know if he thought of qubits, but his main interest in quantum computing was for simulating quantum physics and systems.