For more information on transpiling in Qiskit, see this StackOverflow post.
The previous answer captures most of the basic ideas. Transpiling at its most basic is converting one set of gate operations to an equivalent set of gate operations. This might involve approximating certain non-native gates which cannot be implemented directly on a given quantum computer. Using the Solovay-Kitaev Theorem, we know there is a way to approximate any arbitrary unitary gate using a set of basis gates, which you can read about in Nielsen & Chuang 4.5, and Appendix 3.
Refactoring gates into some other set of equivalent gates might involve something like changing a controlled-Z gate into a CNOT-gate conjugated by Hadamards if there is no implementation of a controlled-Z gate on a particular quantum computer. It might involve doing the reverse if one wishes to reduce gate counts. Reducing gate counts is often important because circuit depth increases runtime which leads to "decoherence", and the introduction of errors due to errors in the gates themselves.
To understand the mathematics behind transpiling, it's best to just look at the linear algebra and write out the matrix representations of the gates to prove they are equal (or at least approximately equal within some error bounds). In general, optimizing a quantum algorithm via transpiling is a "QMA-complete" problem, which is a sort of quantum version of NP-complete problems. So, in general it is not an easy thing to do. However, you can expect some cases to be easier. This is discussed more in Nielsen & Chuang. You might also want to read up on verifiable optimizations using automated theorem provers. This may prove important in the future since verifying quantum circuits without some formal proof system is in general too complex to do on a classical computer and requires a quantum computer in some cases.
One might also try a machine learning approach that takes into account the graph stricture of a quantum circuit as well as the semantics, but I haven't found any examples of this yet.
The transpiler in Qiskit at present doesn't seem to function well consistently, and I'm currently investigating exactly why this is. As of now (May 14, 2020), the StackOverflow link I gave above gives several counterexamples of gates that cannot be transpiled in Qiskit. Some of these include a permutation matrix on 3-qubits, and randomly generated unitary gates on 3 or more qubits using the
random_unitary() function in Qiskit.
I hope this helps you understand transpilers somewhat better.