The description of carrier dynamics in spatially confined semiconductor nanocrystals (NCs), which have enhanced electron-hole and exciton-phonon interactions, is a great challenge for modern computational science. These NCs typically contain thousands of atoms and tens of thousands of valence electrons with discrete spectra at low excitation energies, similar to atoms and molecules, that converge to the continuum bulk limit at higher energies. Computational methods developed for molecules are limited to very small nanoclusters, and methods for bulk systems with periodic boundary conditions are not suitable due to the lack of translational symmetry in NCs. This perspective focuses on our recent efforts in developing a unified atomistic model based on the semiempirical pseudopotential approach, which is parameterized by first-principle calculations and validated against experimental measurements, to describe two of the main nonradiative relaxation processes of quantum confined excitons: exciton cooling and Auger recombination. We focus on the description of both electron-hole and exciton-phonon interactions in our approach and discuss the role of size, shape, and interfacing on the electronic properties and dynamics for II-VI and III-V semiconductor NCs.