We show that a static extremal black hole in 4D gravity, non-linearly coupled to a massive Stückelberg scalar, can exhibit a quantum phase transition due to electric charge variations. The critical point exists only if the spacetime possesses cosmological constant (positive or negative) and the mass of the scalar is non-vanishing. In that case, around the critical point, two solutions coexist: Reissner-Nordström (A)dS black hole and the hairy (A)dS black hole. A near-critical expansion reveals that the entropy and its derivative are continuous functions at the critical point. Obtained results are analytical and based on the entropy function formalism.