Natural type selection in adaptive lossy compression

Ram Zamir, Kenneth Rose

Research output: Contribution to journalArticlepeer-review

Abstract

Consider approximate (lossy) matching of a source string ∼P, with a random codebook generated from reproduction distribution Q, at a specified distortion d. Recent work determined the minimum coding rate R 1 = R(P, Q, d) for this setting. We observe that for large word length and with high probability, the matching codeword is typical with a distribution Q 1 which is different from Q. If a new random codebook is generated ∼Q 1, then the source string will favor codewords which are typical with a new distribution Q 2, resulting in minimum coding rate R 2 = R(P, Q 1, d), and so on. We show that the sequences of distributions Q 1, Q 2, ... and rates R 1, R 2, ..., generated by this procedure, converge to an optimum reproduction distribution Q*, and the rate-distortion function R(P, d), respectively. We also derive a fixed rate-distortion slope version of this natural type selection process. In the latter case, an iteration of the process stochastically simulates an iteration of the Blahut-Arimoto (BA) algorithm for rate-distortion function computation (without recourse to prior knowledge of the underlying source distribution). To strengthen these limit statements, we also characterize the steady-state error of these procedures when iterating at a finite string length. Implications of the main results provide fresh insights into the workings of lossy variants of the Lempel-Ziv algorithm for adaptive compression.

Original languageEnglish
Pages (from-to)99-111
Number of pages13
JournalIEEE Transactions on Information Theory
Volume47
Issue number1
DOIs
StatePublished - Jan 2001

Keywords

  • Adaptive compression
  • Alternating optimization
  • Approximate string matching
  • Blahut-Arimoto algorithm
  • Lempel-Ziv coding
  • Rate-distortion
  • Typical sequences
  • Universal coding

Fingerprint

Dive into the research topics of 'Natural type selection in adaptive lossy compression'. Together they form a unique fingerprint.

Cite this