We develop an exact formalism for performing first-principles calculations for insulators at fixed electric polarization. As shown by Sai, Rabe, and Vanderbilt (SRV) [Phys. Rev. BPRBMDO0163-1829 66, 104108 (2002)10.1103/PhysRevB. 66.104108], who designed an approximate method to tackle the same problem, such an approach allows one to map out the energy landscape as a function of polarization, providing a powerful tool for the theoretical investigation of polar materials. We apply our method to a system in which the ionic contribution to the polarization dominates (a broken-inversion-symmetry perovskite), one in which this is not the case (a III aV semiconductor), and one in which an additional degree of freedom plays an important role (a ferroelectric phase of KNO3). We find that while the SRV method gives rather accurate results in the first case, the present approach provides important improvements to the physical description in the latter cases.