440

IEEE COMMUNICATIONS LETTERS, VOL. 6, NO. 10, OCTOBER 2002

Compression of Binary Sources With Side Information at the Decoder Using LDPC Codes Angelos D. Liveris, Student Member, IEEE, Zixiang Xiong, Senior Member, IEEE, and Costas N. Georghiades, Fellow, IEEE

Index Terms—Channel coding, distributed source coding, LDPC codes, Slepian–Wolf theorem.

(“Wyner’s scheme” [10]) for the case where one of the two correlated sources is available losslessly at the joint decoder (asymmetric case). The application of LDPC codes to this compression problem with side information is based on two points. First, it is straightforward to modify the conventional message passing LDPC decoder to take the syndrome information into account in the binary case considered here. Second, all LDPC code design techniques can be applied to distributed source coding producing simulation results better than any turbo coding scheme suggested so far.

I. INTRODUCTION

II. SYSTEM, CORRELATION CHANNEL AND SYNDROMES

CCORDING to the Slepian–Wolf theorem [1], the lossless compression of the output of two correlated sources that do not communicate their outputs to each other can be as efficient as if they communicated their outputs. This is true when their compressed outputs are jointly decompressed at a decoder. Such systems and their achievable rates are shown in [1]–[3]. Practical schemes exploiting the potential of the Slepian– Wolf theorem were recently introduced based on channel codes, like block and trellis codes [4]. More advanced schemes with better results were subsequently proposed based on the more powerful turbo codes [2], [3], [5]–[7]. However, despite all the different approaches proposed so far, the problem of relating the Slepian–Wolf turbo code design with the turbo code design in “conventional” channel coding has not been solved yet, so the large amount of work already available on turbo codes cannot be exploited. Low-density parity-check (LDPC) codes seem to be more suited for such an application. Their application to the Slepian– Wolf problem was first suggested in [8] in a more general and theoretical context and nowadays they appear to be the most powerful channel codes [9]. To compress a sequence of source output bits, the corresponding syndrome is determined using the sparse parity check matrix as proposed in [8]. But the symmetric case considered in [8] could not be linked to the already available LDPC code design results in a straightforward way. Our main contribution in this letter is to show how LDPC codes can be employed when viewing the problem using an equivalent channel and applying the syndrome approach

We consider the system of Fig. 1 with the following assumptions, which are used for the rest of this letter. , , where • ’s and ’s are i.i.d. equiprobable binary random the variables. and are correlated with . • is available losslessly at the joint decoder and we try • as efficiently as possible. Since the rate to compress is its entropy bits, used for is [1] the theoretical limit for lossless compression of

Abstract—We show how low-density parity-check (LDPC) codes can be used to compress close to the Slepian–Wolf limit for correlated binary sources. Focusing on the asymmetric case of compression of an equiprobable memoryless binary source with side information at the decoder, the approach is based on viewing the correlation as a channel and applying the syndrome concept. The encoding and decoding procedures are explained in detail. The performance achieved is seen to be better than recently published results using turbo codes and very close to the Slepian–Wolf limit.

A

Manuscript received April 15, 2002. The associate editor coordinating the review of this letter and approving it for publication was Prof. M. Fossorier. This work will be presented at IEEE Globecom’02, Taipei, Taiwan, R.O.C., November 2002. The authors are with the Electrical Engineering Department, Texas A&M University, College Station, TX 77843 USA (e-mail: [email protected]). Digital Object Identifier 10.1109/LCOMM.2002.804244.

. With the three assumptions above, we end up with the problem of compression of an equiprobable memoryless binary source with side information. An equivalent way to view the system of Fig. 1 in order to allow the use of channel codes is by modeling the correlation and with a binary symmetric channel (BSC) with between will be the input to the channel and crossover probability . its output as shown in Fig. 2. Then the compressed version of , i.e., , can be used to make look like a codeword of a channel code. binary block code in this binary case, Using a linear distinct syndromes, each indexing a set of there are binary words of length . We call the linear block code (all-zeros sets are disjoint syndrome set) the original code. All the and in each set the Hamming distance properties of the original code are preserved, i.e., all codes have the same performance over the binary symmetric correlation channel. In compressing, a sequence of input bits is mapped into its corresponding syndrome bits. Thus, the compression ratio achieved with . This approach, known as “Wyner’s this scheme is : scheme” [10] for some time, has so far only been used in [4] for the design of simple codes.

1089-7798/02$17.00 © 2002 IEEE

LIVERIS et al.: COMPRESSION OF BINARY SOURCES USING LDPC CODES

Fig. 1.

441

System for compression with side information. TABLE I COMPARISON OF [5] WITH OUR LEFT REGULAR LDPC CODE RESULTS

Fig. 2. Equivalent correlation channel for compression with side information.

III. ENCODING AND DECODING WITH LDPC CODES A low-density parity-check (LDPC) code is determined by its parity-check matrix or, equivalently by its bipartite graph. An ensemble of LDPC codes is described by the degree distribution and [11]–[13]. The bipartite graph is polynomials used in the message-passing decoding algorithm [9], [11]–[13].

The values are assigned to the corresponding according to the connections in the bipartite graph and are then used to do the processing at the check nodes. From the “tanh rule” and the syndrome information, the LLR sent from the th check node along the th edge is

A. Encoding

(3)

Given , to encode, i.e., compress, an arbitrary binary input sequence, we multiply with [8] and find the corresponding . Equivalently, in the bipartite syndrome graph, this can be viewed as binary addition of all the variable node values that are connected to the same check node.

, . The inclusion of the factor accounts for the syndrome information. for all edges in the bipartite graph, Now which can be used to start a new iteration and estimate from

B. Decoding from The decoder must estimate the -length sequence -long syndrome and the corresponding -length its sequence . We use the following notation: , , are the current values — , and , respectively, corresponding to the th variable of node ; , , is the degree of ; — , , , is — the log-likelihood ratio (LLR) sent along the th edge from (to) ; , is the value of — corresponding to the th check node , i.e., the th syndrome component; , , is the degree of ; — , , , — is the LLR sent along the th edge from (to) . Setting (1) , the th variable node

,

along the

, the LLR sent from th edge is

, where initially

if (4) if

.

IV. SIMULATION RESULTS AND COMPARISON WITH TURBO-CODES A. Simple Left Regular Codes We first simulated left regular LDPC codes and compared the results with those of turbo-codes [5] using approximately , i.e., the same parameters. The codeword length is equal to the interleaver length of [5]. More than 2000 blocks were transmitted without a single error after 40 iterations of the message passing algorithm. As for the LDPC code distribution, and , where varies so that the code rate changes. The results are given in Table I together with those of and are the rates in compressed bits per information [5]. and , respectively, and so in our case always bit used for . From the table, it is clear that even these simple LDPC codes show small gains over the turbo code performance.

(2)

B. Improved Irregular Code Design

.

Much better results are expected from LDPC codes with improved irregular code design [9], [11]–[13]. We use the simplest

442

IEEE COMMUNICATIONS LETTERS, VOL. 6, NO. 10, OCTOBER 2002

TABLE II COMPARISON OF [3] WITH OUR IRREGULAR LDPC CODE RESULTS

compared with those of [3]. For the rate 1/2 codes we considered the results of the LDPC code of [12, example 2], whereas for the other two cases we designed a rate 3/4 and a rate 7/8 LDPC code respectively using the method of [11], obtaining the distributions

(7) (8) for

and

Fig. 3. Simulation results for the regular (3,6) and three irregular rate 1/2 LDPC codes of length n = 10 and n = 10 for 100 iterations in the decoder. The irregular codes marked “awgn” both have the distribution of (5) and (6), while the irregular marked “bsc” is given in [12, Example 2]. For each point either 100 frame errors or 5 10 bits were simulated.

(9)

2

good irregular code of [13], 1 i.e., the rate 1/2 code with the degree distribution

(5) (6) This irregular code was simulated together with the regular (3,6) code in Fig. 3. The codeword (frame) length was for the regular code and and for the irreg, ular code (marked “awgn”). The bit error rate (BER) for , was measured after 100 iterations of the decoding algorithm. The Slepian–Wolf theoretical limit of 0.5 bits and the best turbo code performance reported for this code rate [3] are also shown in Fig. 3. The second irregular code, marked “bsc” in Fig. 3, is the rate 1/2 LDPC code given in [12, Example 2], which has been optimized for the binary symmetric channel bits [12], (BSC). The threshold for this code is also shown in Fig. 3. In Fig. 3 even the regular (3,6) code slightly outperforms the turbo coding schemes of [2], [3], which use greater or equal codeword length, while the irregular code of (5) and (6) almost . halves the gap to the Slepian–Wolf limit, even with Further irregular LDPC code design for the binary symmetric channel yields slightly better results as the difference between the “awgn” and “bsc” curves of Fig. 3 shows. However, regular codes could not outperform the codes of [3] for higher compression, i.e., for 4 : 1 and 8 : 1 compression only good irregular codes could achieve slightly higher compression. In Table II our results for length 10 irregular LDPC codes are 1The irregular codes in [13] have been optimized for the additive white Gaussian noise (AWGN) channel, but they are expected to perform better than regular or nonoptimized irregular codes over the binary symmetric channel (BSC) as well [13].

(10) . For the design of these two LDPC codes we for considered right regular codes, allowed a maximum left node degree of 50, and allowed upper-bounded of the percentage of degree-2 variable nodes. All irregular LDPC results shown in for 5 10 simulated bits and Table II assume a 100 iterations. REFERENCES [1] D. Slepian and J. K. Wolf, “Noiseless coding of correlated information sources,” IEEE Trans. Inform. Theory, vol. IT-19, pp. 471–480, July 1973. [2] J. Bajcsy and P. Mitran, “Coding for the Slepian–Wolf problem with turbo codes,” in Proc. IEEE Globecom ’01, vol. 2, Nov. 2001, pp. 1400–1404. [3] A. Aaron and B. Girod, “Compression with side information using turbo codes,” in Proc. IEEE DCC, Apr. 2002, pp. 252–261. [4] S. Pradhan and K. Ramchandran, “Distributed source coding using syndromes (DISCUS): Design and construction,” in Proc. IEEE DCC, Mar. 1999, pp. 158–167. [5] J. Garcia-Frias and Y. Zhao, “Compression of correlated binary sources using turbo codes,” IEEE Comm. Letters, vol. 5, pp. 417–419, Oct. 2001. [6] P. Mitran and J. Bajcsy, “Turbo source coding: A noise-robust approach to data compression,” in Proc. IEEE DCC, Apr. 2002, p. 465. [7] A. D. Liveris, Z. Xiong, and C. N. Georghiades, “A distributed source coding technique for correlated images using turbo-codes,” IEEE Commun. Lett., vol. 6, pp. 379–381, 2002. [8] T. Murayama, “Statistical mechanics of linear compression codes in network communication,” Europhysics Lett., 2001. preprint. [9] S.-Y. Chung, G. D. Forney, T. J. Richardson, and R. Urbanke, “On the design of low-density parity-check codes within 0.0045 db of the Shannon limit,” IEEE Commun. Lett., vol. 5, pp. 58–60, Feb. 2001. [10] R. Zamir, S. Shamai (Shitz), and U. Erez, “Nested linear/lattice codes for structured multiterminal binning,” IEEE Trans. Inform. Theory, vol. IT-48, pp. 1250–1276, June 2002. [11] S.-Y. Chung, T. J. Richardson, and R. Urbanke, “Analysis of sum– product decoding of low-density parity-check codes using a Gaussian approximation,” IEEE Trans. Inform. Theory, vol. IT-47, pp. 657–670, Feb. 2001. [12] T. J. Richardson, M. A. Shokrollahi, and R. L. Urbanke, “Design of capacity-approaching irregular low-density parity-check codes,” IEEE Trans. Inform. Theory, vol. IT-47, pp. 619–637, Feb. 2001. [13] S.-Y. Chung, “On the construction of some capacity-approaching coding schemes,” Ph.D. dissertation, Massachusetts Institute of Technology, 2000.