On-The-Fly Stochastic Codebook Re-generation for Sources with Memory

Ahmed Elshafiy, Mahmoud Namazi, Ram Zamir, Kenneth Rose

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

This paper proposes a generalized stochastic mechanism for codebook generation in lossy coding settings for sources with memory. Earlier work has shown that the rate-distortion bound can be asymptotically achieved for discrete memoryless sources by a “natural type selection” (NTS) algorithm. In iteration n, the distribution that is most likely to produce the types of a sequence of K codewords of finite length ` that “dmatch” a respective sequence of K source words of length `, (i.e., which satisfy the distortion constraint), is used to regenerate the codebook for iteration n + 1. The resulting sequence of codebook generating distributions converges to the optimal distribution Q that achieves the rate-distortion bound for the memoryless source, asymptotically in `, K, and n. This work generalizes the NTS algorithm to account for sources with memory. The algorithm encodes m`-length source words consisting of ` vectors (or super-symbols) of length m. We show that for finite m and `, the sequence of codebook reproduction distributions Q0,m,`, Q1,m,`, . . . (each computed after observing a sequence of K d-match events) converges to the optimal achievable distribution Qm,` (within a set of achievable distributions determined by m and `), asymptotically in K and n. It is further shown that Qm,` converges to the optimal reproduction distribution Q that achieves the rate-distortion bound for sources with memory, asymptotically in m and `.

Original languageEnglish
Title of host publication2020 IEEE Information Theory Workshop, ITW 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728159621
DOIs
StatePublished - 11 Apr 2021
Event2020 IEEE Information Theory Workshop, ITW 2020 - Virtual, Riva del Garda, Italy
Duration: 11 Apr 202115 Apr 2021

Publication series

Name2020 IEEE Information Theory Workshop, ITW 2020

Conference

Conference2020 IEEE Information Theory Workshop, ITW 2020
Country/TerritoryItaly
CityVirtual, Riva del Garda
Period11/04/2115/04/21

Funding

FundersFunder number
National Science FoundationCCF-1909423
Bonfils-Stanton Foundation2018690

    Keywords

    • Natural Type Selection
    • Random Codebook
    • Rate-Distortion function
    • String Matching

    Fingerprint

    Dive into the research topics of 'On-The-Fly Stochastic Codebook Re-generation for Sources with Memory'. Together they form a unique fingerprint.

    Cite this