Why Are Gauge Theories Successful?

For many years I felt like tearing my hair out whenever I saw a Lagrangian whilst trying to get some layperson’s insight into what other physicists were doing, particularly with BSM physics. Not that I had any problem with the mathematical correctness of what was being presented – it’s physical grounding just seemed thoroughly mysterious. As Roger Penrose says somewhere in “The Road to Reality” that Lagrangian formulations of a theory are “a dime a dozen” (not exactly the words he used: but he meant that you can dream up a Lagrangian account of any physical theory). How on Earth does one dream up a Lagrangian? It is generally hard if not impossible to look at the terms in a Lagrangian and say “that one means such and such” as you can with many (not all, mind you) physical theories. Penrose made the cryptic comment that the standard model would look thoroughly “contrived” if it weren’t for its experimental grounding. I read “contrived” as meaning “not at all physically obvious” but also “if it weren’t for …” implied that the experimental results said this was just how it had to be. I wondered what kind of experimental results would motivate something so abstract as some of the Lagrangians I came across in a way as powerfully as Penrose implied.

Then the following dawned on me:

Lagrangian dynamics + Noether’s Theorem = A tool for theorists to encode the observations of experimentalists into a candidate theory for the theorists to work from

Noether’s theorem is of course about Lagrangians, their continuous symmetries and corresponding conserved quantities, exactly one for each continuous symmetry, whose conservation can be described by a corresponding continuity equation. So, if we experimentally find that there are some measured, real-valued quantities which are conserved throughout experiments, let’s say “twanglehood”, “bloobelship” and “thwarginess”, and then a possible theory is one derived from a Lagrangian which is explicitly constructed with one continuous symmetry for each of these. Moreover we might be lucky to also have three observed continuous symmetries as well. This is now a really strong experimental motivation: we must now write down Lagrangian with the three observed symmetries and try to fit each continuity equation implied by Noether’s theorem to “twanglehood”, “bloobelship” and “thwarginess”, in keeping with whatever else we can experimentally learn about these three.

Once I understood this, then the other mystery melted away. This one drove me mad:

Why do we want physical theories with gauge symmetry – i.e. redundancy in them? Surely physics aims to make things as simple as it can, particularly if the gauge symmetry is not an experimentally initially obvious symmetry of the system?

Of course, in the Lagrangian formulation symmetry is needed to beget conservation, so we take on “redundancy” – gauge symmetry – to express that conservation mathematically in a gauge theory.

Note that these principles are still a “hunch”. There is no reason why conservation implies invariance of a Lagrangian with respect to a symmetry – Noether’s theorem is the other way around. But Noether’s theorem is one way wherein conservation can arise, so we “suck and see” by using this principle to derive a theory and seeing how it stacks up with experiment. With particle physics, it seems that his hunch has been rather successful.

Afternote: I deliberately used the word “continuous” rather than differentiable symmetry: you don’t need to assume the latter. A “continuous” symmetry implies a Lie group of symmetries, and the Montgomery, Gleason and Zippin solution to Hilbert’s fifth problem shows that $C^0$ assumptions in Lie theory imply an analytic, i.e. $C^\omega$ manifold.