We present an algorithm for reconstructing a sample surface potential from its Kelvin probe force microscopy (KPFM) image. The measured KPFM image is a weighted average of the surface potential underneath the tip apex due to the long-range electrostatic forces. We model the KPFM measurement by a linear shift-invariant system where the impulse response is the point spread function (PSF). By calculating the PSF of the KPFM probe (tip+cantilever) and using the measured noise statistics, we deconvolve the measured KPFM image to obtain the surface potential of the sample.The reconstruction algorithm is applied to measurements of CdS-PbS nanorods measured in amplitude modulation KPFM (AM-KPFM) and to graphene layers measured in frequency modulation KPFM (FM-KPFM). We show that in the AM-KPFM measurements the averaging effect is substantial, whereas in the FM-KPFM measurements the averaging effect is negligible.