Non-Gaussian Sources

Inherent in the treatment thus far is that the source has a Gaussian distribution, for if the source is not Gaussian then the high probability region is not a sphere, but some other shape [46], and the wrapped SVQ cannot be effectively used. This section presents a method to obtain the performance above for any memoryless source. The method consists of transform coding the source. Typically, transform coding is done to remove dependencies between consecutive samples of the source; here, it is used to change the distribution of the source, which may or may not already be i.i.d., to be roughly Gaussian and i.i.d., so that wrapped SVQ may still be used. This same intuition was used in [10] to quantize an arbitrary source and obtain distortion performance that approximates that of a scalar quantizer for a Gaussian source. Unlike the approach in [10], in this section the source is transformed in blocks, instead of using FIR filters.

Let $Q(\cdot)$ be the output of any $k$-dimensional vector quantizer. Let

$$X \equiv \begin{pmatrix} X_1 & X_{m+1} & & X_{(k-1)m+1} \\ \vdots & \vdots & \cdots & \vdots \\ X_m & X_{2m} & & X_{km} \end{pmatrix}$$

where $X_i \in {\mathbb{R}}$ has an arbitrary distribution. Let ${\cal H}_m$ be a Hadamard matrix of order $m$, i.e., an $m\times m$ matrix with $+1$ and $-1$ entries only such that ${\cal H}_m^T {\cal H}_m = mI$. Such matrices are known to exist when the order is any power of 2, and for many other orders as well.Let $H_m \equiv (1/\sqrt{m}){\cal H}_m.$ Given $X$, the vector quantizer output is: $H_m^TQ(H_mX)$. If $Y \equiv H_mX$, $\hat Y \equiv Q(Y)$, and $e \equiv \hat Y - Y$, then it follows that

$$\hat X = H_m^T \hat Y$$

$$= H_m^T (Y + e)$$

$$= H_m^T (H_mX + e)$$

$$= H_m^TH_mX + H_m^Te$$

$$= X + H_m^Te.$$

The end-to-end distortion of this system is

$$E[\Vert\hat X - X\Vert^2] = E[(\hat X - X)^T(\hat X - X)]$$

$$= E[(H_m^Te)^T(H_m^Te)]$$

$$= E[e^TH_mH_m^Te]$$

$$= E[e^Te]$$

$$= E[(\hat Y - Y)^T(\hat Y - Y)]$$

$$= E[\Vert\hat Y - Y\Vert^2]$$

Thus, the end-to-end distortion of the system is equal to the distortion due to the quantization of the intermediary signal $Y$ alone. Most importantly, the Hadamard transform modifies the distribution of the input to the vector quantizer. A row $Y_i$ of $Y$ is a $k$-vector, each component of which is the sum of $m$ different samples (or their negation) from $\{X_i\}$; hence, as $m \rightarrow \infty$ the probability distribution of each component of $Y_i$ approaches the Gaussian distribution, by the central limit theorem. Thus, the internal $k$-dimensional quantizer $Q$ may be optimized with respect to the Gaussian distribution, even if $k$ is fixed and small.