We calculate the entropy of a static extremal black hole in 4D gravity, non-linearly coupled to a massive Stueckelberg scalar. We find that the scalar field does not allow the black hole to be magnetically charged. We also show that the system can exhibit a phase transition due to electric charge variations. For spherical and hyperbolic horizons, the critical point exists only in presence of a cosmological constant, and if the scalar is massive and non-linearly coupled to electromagnetic field. On one side of the critical point, two extremal solutions coexist: Reissner-Nordström (A)dS black hole and the charged hairy (A)dS black hole, while on the other side of the critical point the black hole does not have hair. A near-critical analysis reveals that the hairy black hole has larger entropy, thus giving rise to a zero temperature phase transition. This is characterized by a discontinuous second derivative of the entropy with respect to the electric charge at the critical point. The results obtained here are analytical and based on the entropy function formalism and the second law of thermodynamics.