We study charged, static, topological black holes in pure Gauss-Bonnet gravity in asymptotically AdS space. As in general relativity, the theory possesses a unique nondegenerate AdS vacuum. It also admits charged black hole solutions which asymptotically behave as the Reissner-Nordström AdS black hole. We discuss black hole thermodynamics of these black holes. Then we study phase transitions in a dual quantum field theory in four dimensions, with the Stückelberg scalar field as an order parameter. We find in the probe limit that the black hole can develop hair below some critical temperature, which suggests a phase transition. Depending on the scalar coupling constants, the phase transition can be first or second order. Analysis of the free energy reveals that, comparing the two solutions, the hairy state is energetically favorable, thus a phase transition will occur in a dual field theory.