We consider passive joint estimation of the time-difference of arrival (TDOA) and frequency-difference of arrival (FDOA) of an unknown signal at two sensors. The classical approach for deriving the Cramér-Rao bound (CRB) in this context assumes that the signal (as well as the noise) is Gaussian and stationary. As a result, the obtained Fisher information matrix with respect to the TDOA and FDOA is diagonal, implying that the respective estimation errors are uncorrelated (under asymptotic conditions). However, for some specific (non-Gaussian, non-stationary) signals, especially chirp-like signals, these errors can be strongly correlated. In this work we derive a "signal- specific" (or a "conditional") CRB for this problem: Modeling the signal as a deterministic unknown, we obtain a bound which, given any particular signal, can reflect the possible signal-induced correlation between the TDOA and FDOA estimates. We further demonstrate that this bound is instrumental for proper weighting when combining joint TDOA and FDOA estimates from independent intervals.