Axions play a central role in many realizations of large field models of inflation and in recent alternative mechanisms for generating primordial tensor modes in small field models. If these axions couple to gauge fields, the coupling produces a tachyonic instability that leads to an exponential enhancement of the gauge fields, which in turn can decay into observable scalar or tensor curvature perturbations. Thus, a fully self-consistent treatment of axions during inflation is important, and in this work we discuss the perturbative constraints on axions coupled to gauge fields. We show how the recent proposal of generating tensor modes through these alternative mechanisms is in tension with perturbation theory in the in-in formalism. Interestingly, we point out that the constraints are parametrically weaker than one would estimate based on naive power counting of propagators of the gauge field. In the case of non-Abelian gauge fields, we derive new constraints on the size of the gauge coupling, which apply also in certain models of natural large field inflation, such as alignment mechanisms.