Why are orthogonal functions and eigenvalues/functions so important in quantum mechanics?

I gave this answer on Physics Stack Exchange to the following question:

The mathematics and physics we have studied so far at university are heavily focused around the idea of orthogonal functions, orthogonality, sets of solutions, eigenvalues and eigenfunctions.

Why are we so interested in these properties? What are the conceptual aspects of them, mainly in quantum mechanics?

and the answer, slightly edited, is as follows:

They are important for several reasons:

1. Orthogonal functions arise naturally in the study of Sturm-Liouville theory which includes many classical and quantum system mathematical models;

2. More generally, it is the class of normal operators (and an important special case of these, to wit self adjoint operators) which the spectral theorem most readily works and is most complete for. The eigenvectors of such operators are always orthogonal. The “Diagonalising” an operator in any linear system theory is an important step for understanding – it means we can decouple the operator’s action into the sum of its action on altogether uncoupled eigenvectors. It’s an important step in “untangling” a highly coupled problem. In the context of when the Hilbert space concerned is a function space, the relevant Sturm-Liouville theory, e.g. for the quantum harmonic oscillator shows that the linear space of all “practical”, normalisable quantum states is spanned by discrete eigenfunctions. In other words, the Hilbert space’s dimension is *countably* infinite, even though we are dealing with spaces of continuous functions and you might intuitively think the dimension cardinality might be $\aleph_1$, and that’s just too scary to deal with!

3. We deal often with two important conservation laws: conservation of energy and conservation of probability. These conservation laws are most readily expressed if the basis for the relevant state space is orthogonal – it means that energy, power or probability as appropriate is simply the $\mathbf{L}^2$ length of any vector. We don’t have to manage cross coupling terms in our inner product space. Whether it be functions or Cartesian bases for three dimensional Eucliean space, projections and resolution into basis superpositions are always heaps easier and clearer if the basis is orthogonal. You’d be a sucker for punishment if you did an everyday geometric problem in $\mathbb{R}^3$ with a general, linearly independent but nonorthogonal basis, even though this can certainly be done. Exactly the same intellectual work minimalisation principles apply to  functions spaces as much as they do to $\mathbb{R}^3$. Energy- or probability-conservative system transformations are then unitary and so on and so forth.