This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2008 proceedings.

Swifter: Chunked Network Coding for Peer-to-Peer Content Distribution Jinbiao Xu, Jin Zhao, Xin Wang+, Xiangyang Xue Department of Computer Science and Engineering Fudan University Shanghai, China + Corresponding author: [email protected] Abstract—The benefit of network coding with respect to simplifying scheduling overhead for content distribution has been extensively studied in previous literature. However, the complexity of network coding increases as the content size scales up. In this paper, we study the tradeoff between scheduling overhead and coding overhead. To this end, we propose Swifter, a P2P content distribution scheme, which employs local-rarest-first segment scheduling and chunked network coding algorithms. In Swifter, content is divided into segments, which are further divided into blocks. Each peer schedules a local-rarest segment request from its neighbors. Network coding is then used for generating a reply block within the requested segment. Leveraging our real-world implementation and experiments, we find that Swifter has low coding overhead and can reduce average download time by up to 40% compared to existing work. Keywords-network content distribution

coding;

local-rarest-first;

peer-to-peer;

I. INTRODUCTION Peer-to-peer, or P2P, networks have emerged as a promising approach to large-scale content distribution [4][7][8]. The advantages of P2P networks lie in the flexibility in overlay topology and the scalability in serving capacity. In P2P networks, as any pair of peers can establish a virtual connection, the overlay topology can be constructed at will. Meanwhile, peers can collaborate with each other and thus the serving capacity is amplified. Despite such important advantages, peer-to-peer content distribution poses the following two significant challenges, especially when it comes to large-scale deployment. First, peer dynamics. P2P networks are dynamic in nature since a peer may arrive, depart, or fail without notice. When some peers are not available, certain blocks may become rare or even unavailable. A peer has to locate the last missing blocks to finish downloading. As a consequence of peer dynamics, such rare blocks will eventually result in longer download time. Second, block reconciliation. Parallel downloading from multiple upstream peers is a typical way of achieving high throughput [4][13]. Since peers have to exchange missing blocks with each other, they need to decide which upstream peer to retrieve a specific block from, which is referred to as block reconciliation problem. Block reconciliation is a nontrivial scheduling task. On the one hand, a peer needs to

schedule blocks from as many other peers as possible to exploit its available downlink bandwidth. On the other hand, different peers need to collaborate appropriately to reconcile the differences among the received blocks. Otherwise, receiving redundant blocks from multiple upstream peers is a waste of bandwidth. In their pioneering work, Ahlswede et al. [1] introduced network coding. The essence of network coding is to allow peers to code their received information before forwarding it to downstream peers. With network coding, a peer need not decide which block to schedule, but simply requests its upstream peer to send a coded block. It thus can decode the original content after receiving enough number of coded blocks. In other words, all the coded blocks are of equal importance. Therefore, block reconciliation is eliminated. Recent study [5] shows that with network coding, there may be an improvement in resilience to peer dynamics on account of high block availability. Though encouraging, network coding introduces excessive computational complexity as it involves coding operations at every peer. The problem is especially true when the content is large in size. Recent work [14] has doubted the feasibility of network coding in P2P content distribution since the coding speed is not tolerable even with a modern CPU. Coding complexity increases dramatically as the number of coding blocks scales up. On the venue to reduce network coding complexity, chunked coding scheme [10] attracts much attention. The basic idea of chunked coding is to divide the file into fixed-size segments (or generations, chunks), and each segment is further divided into blocks. Network coding is then performed on the blocks within each segment. Such idea originated from the previous work [11] by Chou et al., and is referred to as Group Network Coding in Avalanche [12]. Chunked coding can bring two major benefits: 1) reducing computational complexity 2) decreasing the size of encoding vectors. Nevertheless, though chunked coding eliminates block reconciliation within a segment, a peer still need consider which segment’s block to schedule, referred to as segment reconciliation problem. Observing that the dynamics of peer participation, or churn, is an inherent property of P2P systems, we seek a scheduling solution, which has resilience to peer dynamics, to the problem of segment reconciliation. Recall that chunked coding can

978-1-4244-2075-9/08/$25.00 ©2008 IEEE Authorized licensed use limited to: Xiamen University. Downloaded on June 2, 2009 at 10:08 from IEEE Xplore. Restrictions apply.

5603

This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2008 proceedings.

reduce coding complexity at the expense of segment scheduling overhead, in this paper, our objective is to design a chunked coding content distribution system with high scheduling efficiency. To obtain efficient content distribution, a natural designing question arises: how to efficiently schedule all the segments? Inspired by the work [13] which suggests that the local-rarest-first policy outperforms the random policy at block-level, we make use of the local-rarest-first scheduling algorithm at segment-level. Based on the local-rarest-first algorithm and chunked network coding, we propose a novel content distribution system “Swifter”. The core of Swifter is its pull-based segmentscheduling algorithm. After the receiver requesting for the rarest segment, the sender may reply a new encoded block in the corresponding segment. The rarest segment has the smallest number of blocks among the local peers. To find out the rarest segment, peers have to periodically exchange buffer-maps, which actually incur little overhead compared with transmitting the content. To evaluate the performance of Swifter, we have also designed and completed a real-world implementation, henceforth referred to as Swifter, too. In our experience, such scheduling algorithm is efficient even under peer churn. We have also found that computational overhead is quite affordable. To the best of our knowledge, this is the first attempt to combine network coding with the local-rarest-first algorithm. The primary contribution of this paper is two-fold. First, we propose a novel P2P content distribution scheme which employs chunked network coding and local-rarest-first scheduling algorithms to address the problem of peer dynamics and segment reconciliation. Second, we have also implemented a real-world Swifter. Our experimental results confirm that with chunked network coding and local-rarest-first scheduling algorithm, Swifter reduces the average download time, which is an important evaluation metric. The remainder of this paper is organized as follows. Section II discusses related work. In Section III, we present Swifter’s architecture and algorithms. In Section IV, we present main experimental results and our analysis. We conclude this paper in Section V. II.

RELATED WORK

During recent years, there has been a significant increase in the popularity of P2P applications ranging from file-sharing (e.g., Gnutella and FastTrack) to live streaming (e.g., Coolstreaming and PPlive). To improve the efficiency of block reconciliation, previous works have advocated rateless coding or Tornado coding. Byers et al. [7] proposed that each peer randomly sends different Tornado coded blocks on different downstream links. Bloom filter techniques are then used to reduce data redundancy among peers. Many state-of-the-art P2P content distribution systems such as BitTorrent [4] use local-rarest-first scheduling algorithm at block-level. Though combined with peer selection strategy, BitTorrent-like systems might still suffer from the block availability problem when peer churn rate is high [13].

Figure 1.

The architecture overview of Swifter.

Since the work on linear network coding [2] and random network coding [3], there have been efforts in applying network coding to P2P networks. Avalanche [6][12], for example, demonstrated 2-3 times better throughput over noncoding systems. As far as the coding efficiency is concerned, Shojania et al. [16] proposed a paralleled progressive approach, which uses hardware acceleration. Ma et al. [9] proposed a sparse coding method, which only combines a subset of blocks when generating a new block. Niu et al. [5] studied the tradeoff between coding complexity and block resilience using theoretical model and simulation analysis. However, their work does not mention how segment is scheduled. Swifter is more akin to two recent research papers on chunked coding [10][11], both of which propose push-based (i.e., the sender chooses a segment using one of the methods discussed shortly, and then pushes a new encoded block in the corresponding segment to the receiver) schemes. In [11], the sender chooses segments or generations in a sequential order, and the receiver may provide feedback about which segments are no longer needed. However, such scheduling method is somewhat naive and this motivates us to exploit more sophisticated scheduling algorithms in order to improve efficiency. Maymounkov et al. [10] argue that random segment selection is sufficient and that feedback from receivers is not required. However, as the sender can not know which segments have been completely received by the receiver with no feedback, this might result in pushing redundant blocks. Swifter is distinguished from existing works in two important aspects. First, it employs a pull-based approach. Second, the segments are scheduled in local-rarest-first order instead of random or sequential order. III.

SYSTEM OVERVIEW

In this section, we briefly summarize Swifter’s architecture and algorithms. The core of Swifter includes a peer neighbor manager, a segment scheduler, an encoding and decoding module, a dependence checker and a buffer-map manager. Fig. 1 shows the architecture of Swifter. There are two kinds of nodes—server and peer. Server is a special peer which owns the original content, and it is also a tracker [4] acting as a

5604 Authorized licensed use limited to: Xiamen University. Downloaded on June 2, 2009 at 10:08 from IEEE Xplore. Restrictions apply.

This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2008 proceedings.

bootstrapping peer. Peers periodically exchange messages to maintain a self-organized overlay network. There can be only one server and an unlimited number of peers in our system. The neighbor manager can detect arrivals and departures of peers, and it updates peers’ information as well. The buffermap manager handles the exchange of buffer-maps and enables feedback among peers. The dependence checker can examine the encoding vectors in order to avoid receiving linear dependent blocks and it is quite useful for network coding systems despite incuring a bit of delay. Swifter’s core algorithm is a pull-based segment scheduler. The scheduler employs the local-rarest-first algorithm to select a segment’s request. According to the scheduler’s request, the receiver can pull a coded block of the corresponding segment from the sender. Encoding and decoding modules deal with the network coding operations over the outgoing and incoming data. A. Overlay Network Model In P2P networks, each peer has knowledge of the identities of only a few other peers, which is called the neighborhood. We assume that the size of the neighborhood is a small value (e.g., 10-15) and the neighboring relation is symmetric. The neighbor manager maintains the neighborhood to keep track of peer dynamics. Downstream peers are a few neighbors who download data blocks while upstream peers upload. Since excessive concurrent uploads may negatively affect the throughput, we limit the maximum downstream degree to 4. B. Content Distribution Model For practical reasons, content is divided into equal-size blocks. All the peers cooperate in distributing the blocks. A set of blocks can be grouped into a so-called segment or generation. In our implementation, the block size is 256KB. A segment may consist of 16-64 blocks. The receiver can request for a segment and then pull a block in that specific segment from the sender. After assembling all the blocks in one segment, the receiver can reconstruct the original segment. Network coding is performed only within segments. In our pull-based model, peers have to exchange their buffer-maps every few seconds. A peer’s buffer-map contains its information about how many blocks have been buffered in each segment. The buffer-map manager deals with the spread and update of such messages. C. Network Coding Performed in a Segment First we briefly summarize the network coding approach applied in Swifter. Assume that the sender has n coded or uncoded blocks [b1i,b2i,...,bni] in segment i, then it can serve a new coded block in segment i which is a random linear combination of [b1i,b2i,...,bni] in Galois Field(28). Suppose that segment i consists of m blocks, the receiver can not reconstruct segment i until it has received m linear independent blocks [c1i,c2i,...,cmi]. To decode segment i, the receiver has to

Figure 2.

Pseudo-code of Swifter’s local-rarest-first segment selection.

invert the coefficient matrix Ai and then multiply Ai-1 by [c1i,c2i,...,cmi]. Suppose we distribute a file containing S segments. The encoding speed can improve by S-times so that there's no need to use Sparse Code [9]. The decoding process includes two steps— inverting the coefficient matrix and recovering the original file, with speedup of S2-times and S-times respectively. In addition, the size of encoding vectors that need to be transmitted can be reduced by a factor of S. In our experiments, S can be as large as 128. D. Segment-scheduling Algorithm The core issue of segment scheduling is letting the receiver effectively decide which segment to request. To guarantee a high diversity of the segments, we make use of the localrarest-first algorithm working as follows. Each node maintains a list with the numbers of blocks of each segment in every neighbor. They periodically exchange and update this information. Among the segments that still need blocks to be

5605 Authorized licensed use limited to: Xiamen University. Downloaded on June 2, 2009 at 10:08 from IEEE Xplore. Restrictions apply.

This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2008 proceedings.

decoded, the receiver can select the rarest segment based on the aforementioned list, which is the local rarest segment. If there are multiple rarest segments, a segment among them is selected at random. Fig. 2 shows the pseudo-code of the localrarest-first segment selection. We believe that the local-rarest-first algorithm is of advantage to segment scheduling in dynamic networks. The first reason is that the local-rarest-first algorithm can strike a balance in distributing different segments so that rare segments can be eliminated. The second is that the server or seed needn’t stay in the network so long attributing to the segment diversity. In a word, the receiver can fetch a linear independent block more efficiently. Actually, a more ideal definition of the rarest segment is that the segment which has the fewest useful blocks (i.e., be linear independent of the receiver’s blocks) among the neighbors. However, it’s infeasible to find such rarest segment. In BitTorrent, the behavior of the local-rarest-first algorithm can be modified to shorten the initial and final part of download time. When it comes to the first few segments to be downloaded, the receiver selects a segment at random. This is referred to as random first policy. To solve the last missing blocks problem, the local-rarest-first algorithm is switched to the end game mode [15]. At the end of the downloading process, the receiver may send the same request to multiple peers. Since it only happens at the very end, the end game mode does not result in significant overhead. For simplicity, we do not implement such modification in our system. And we believe that the overall system performance is not affected apparently. BitTorrent uses choking and unchoking algorithm [15] to discourage free-riding. Since the focus of this paper is system performance rather than fairness, we have not implemented any incentive mechanism. However, Swifter can be combined with such incentive mechanism as well. This simplification does not hamper Swifter’s performance. IV. EXPERIMENTAL RESULTS In this section we study the performance of the Swifter system presented in Section III, comparing it with two alternative schemes for content distribution. To evaluate the performance of each scheme, we are mainly concerned with the average download time. We experience short download time by making the most of network coding and the local-rarest-first algorithm. We have also implemented two alternative schemes using network coding. One is the random segment selection scheme described in [10]. The sender has no idea of which segments are completely received by the receiver, and just pushes an encoded block in a random segment. After completing downloading the whole file, the receiver will cut off all its downlinks to stop receiving. We would like to refer to it as “RPush” in this paper. The other scheme is a variant of Swifter. We would like to refer to it as “R-Swifter” in this paper. Similar

TABLE I.

MAIN EXPERIMENTAL PARAMETERS

Parameter

Value

File size

512MB

Block size

256KB

Segment size

Varies

The number of peers Peer arrival rate Peer lifetime

30 Poisson distribution (λ = 0.4) Varies

Max upstream degree

4

Max downstream degree

4

Bandwidth of each link

128KB/s

to Swifter, R-Swifter is also pull-based, except that it simply uses random segment selection. Peers periodically exchange buffer-maps so that the receiver has full knowledge of which segments are available at the sender. Both R-Push and RSwifter employ random segment selection, there are, however, apparent differences between them. In R-Push, a peer randomly selects one segment and pushes a coded block within that segment to downstream peers. However, the randomly selected segment may be not innovative for the downstream peer if it has finished the segment. In R-Swifter, a peer randomly selects an unfinished segment, and pulls one coded block from its upstream peer. The experimental results show that using the local-rarestfirst algorithm, Swifter can improve the average download time by at most 40% compared to R-Push and, by at most 6% compared to R-Swifter. A. Experimental Setup First we summarize the experimental setup. The experiment task is to distribute a 512MB file among 30 nodes, which are equipped with Pentium IV 3.0GHz CPU and 2.0GB DDR2 RAM, in our campus network. In our experiments, we only have 30 nodes. However, as Swifter’s overlay management shares the same spirit as existing P2P systems, like BitTorrent, we believe that Swifter can scale up well. The 512MB file is divided into a number of 256KB blocks, which are then grouped into 4MB-16MB segments. Network coding is performed within segments. We also set a degree constraint on each peer. A peer can have at most 4 downstream peers, and it can select at most 4 random upstream peers among its neighbors concurrently. Bandwidth of each link between two nodes is limited to 128KB/s. Before receiving a block, the receiver will check the corresponding encoding vector in case of linear dependence. The receiver may choke an upstream peer for 10 seconds when detecting too much linear dependence. Table I summarizes the main parameters of our experiments. In the experiments, we would like to emphasize the resilience to peer dynamics gained by network coding, so all scenarios of experiments are under high churn rate. We assume that inter-arrival times of peer join events are modeled as a

5606 Authorized licensed use limited to: Xiamen University. Downloaded on June 2, 2009 at 10:08 from IEEE Xplore. Restrictions apply.

This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2008 proceedings.

Figure 3.

Average download time with different lifetime (a) T=100s (b) T=60s.

Poisson distribution (λ = 0.4) and there can be at most 30 concurrent peers. All nodes have a uniform lifetime T = 60s or T = 100s. Every node will rejoin the network unless it completes downloading the whole file. B. Metrics To evaluate the performance of each system, we are concerned with three metrics—average download time, maximum download time and server upload. Average download time is an important metric reflecting the system performance. Maximum download time is the worst case of peer download times. Server upload reflects the total contributed content of server in collaborative content distribution. The smaller are the above metrics, the better performance we have. C. Average Download Time Fig. 3 shows the average download time in two scenarios with peer life time T = 60s and T = 100s. In both scenarios, the average download time decreases along with the increase of segment size. Actually, as segmnet size increases, encoding complexity increases as well. However, we experience low CPU usage (less than 5%) in all experiments. So we can infer that computational overhead does not affect the overall system performance. The reason for the decrease of average download time lies in network coding itself. As segment size increases, network coding helps more in increasing block diversity, and we need to schedule fewer segments. Each peer can therefore download useful blocks with less effort. In both scenarios, Swifter improves the average download time by 20%-40% compared to R-Push. We have also noticed that in R-Push, peers may receive useless blocks (i.e., without feedback, a peers may still receive blocks in its completely finished segments) and the block size is considerable large. On the other hand, R-Swifter uses the same random segment selection algorithm but enables feedback. Its average download

Figure 4. Maximum download time with different lifetime (a) T=100s (b) T=60s.

time has also a drastic decrease compared to R-Push. It seems that though feedback may result in communication overhead, it greatly improves the scheduling efficiency of the system. What’s more, Swifter can outperform R-Swifter by 6% when segment size is 16MB. The reason for this improvement is that the local-rarest-first algorithm can increase segment diversity over all the peers, especially in dynamic networks, so peers can always request available segments. This justifies the advantage of local-rarest-first segment selection over random segment selection. And we believe that with longer duration of distribution, the gap between Swifter and R-Swifter would be larger. Another interesting feature we want to address is that as peer churn rate increases (lifetime T decreases), the average download time of Swifter and R-Swifter just increases slightly, and that of R-Push increases by at most 7%. With segment size 16MB, Swifter even has a short average download time less than 1500s. It seems that with network coding, the three systems we examined have much resilience to peer churn. The result is not surprising because with network coding, peers can find useful blocks more easily. D. Maximum Download Time To further confirm our conclusions, we have also examined the maximum download time. Fig. 4 shows the maximum download time for the three schemes. It is consistent with the average download time. In fact, the maximum download time is much larger than the average. However, as segment size increases, the absolute gap between the maximum and the average time has a drastic decrease. It’s also because with the increase of segment size, the total number of segments decreases so that the segment scheduling overhead imposes smaller impact upon all peers. In all scenarios, Swifter has a smaller maximum download time than both R-Swifter and RPush. This suggests that the local-rarest-first segment scheduling algorithm not only helps in the overall system performance, but in the worst case.

5607 Authorized licensed use limited to: Xiamen University. Downloaded on June 2, 2009 at 10:08 from IEEE Xplore. Restrictions apply.

This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2008 proceedings.

REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] Figure 5. Server upload with different lifetime (a) T=100s (b) T=60s.

E. Server Upload Fig. 5 shows the upload of server. We have noticed that the server is always of full degree (i.e., 4 concurrent downstream peers) during the distribution process without apparent impact of high churn rate, and the server upload increases along with the average download time. The reason is that the server has the complete file, and peers are much likely to request blocks from the server. Interestingly, as to Swifter, the server upload is always less than 0.92GB (recall that the file size is 0.5GB). We can infer that among the blocks distributed by the server, there is a great diversity at segment level. While with the random segment selection, it achieves less diversity than the localrarest-first counterpart. V. CONCLUSION In this paper, we have proposed and implemented Swifter, a peer-to-peer content distribution system using both chunked network coding and local-rarest-first segment scheduling algorithms. From our empirical results, we conclude that network coding has advantages in content distribution over a dynamic network. In addition, the local-rarest-first segment scheduling algorithm helps in solving the segment reconciliation problem. We believe that our findings bring insights towards efficient large-scale P2P content distribution. We conclude this paper by claiming the experimental results that Swifter has high performance with respect to short download time and low computational cost.

[9]

[10] [11] [12] [13] [14] [15] [16]

R. Ahlswede, N. Cai, S. R. Li, and R. W. Yeung, “Network Information Flow,” IEEE Transactions on Information Theory, vol. 46, no. 4, pp. 1204–1216, July 2000. S. R. Li, R. W. Yeung, and N. Cai, “Linear Network Coding,” IEEE Transactions on Information Theory, vol. 49, pp. 371, 2003. T. Ho, R. Koetter, M. Medard, D. Karger, and M. Effros, “The Benefits of Coding over Routing in a Randomized Setting,” in Proc. of International Symposium on Information Theory (ISIT 2003), 2003. “BitTorrent,” http://bittorrent.com. Di Niu, and Baochun Li, “On the Resilience-Complexity Tradeoff of Network Coding in Dynamic P2P Networks,” in Proc. 15th IEEE International Workshop on Quality of Service (IWQoS 2007), 2007. C. Gkantsidis, and P. Rodriguez, “Network coding for large scale context distribution,” in Proc. IEEE INFOCOM 2005, March 2005. J. Byers, J. Considine, M. Mitzenmacher, S. Rost: “Informed Content Delivery Across Adaptive Overlay Networks,” In: Proc. of ACM SIGCOMM 2002, August 2002. C. Gkantsidis, J. Miller, and P. Rodriguez, ‘‘Comprehensive View of a Live Network Coding P2P System,’’ in Internet Measurement Conference (IMC 2006), 2006. G. Ma, Y. Xu, M. Lin, and Y. Xuan, “A content distribution system based on sparse linear network coding,” in Proc. 3rd Workshop on Network Coding, Theory, and Applications (NETCOD 2007), January, 2007. P. Maymounkov, N. J.A. Harvey, and D. S. Lun, “Methods for Efficient Network Coding,” in Proc. Allerton Conference on Communication, Control, and Computing, 2006. P.A. Chou, Y. Wu, and K Jain, “Practical network coding,” in Proc. Allerton Conference on Communication, Control, and Computing, 2003. C. Gkantsidis, J. Miller, and P. Rodriguez, “Anatomy of a P2P content distribution system with network coding,” in Proc. 5th International Workshop on Peer-to-Peer Systems (IPTPS 2006), 2006. A.R. Bharambe, C. Herley, and V.N. Padmanabhan, “Analysing and improving bittorrent performance,” in Proc. IEEE INFOCOM 2006, Barcelona, April 2006. M. Wang, and B. Li, “How practical is network coding,” in Proc. 14th IEEE International Workshop on Quality of Service (IWQoS 2006), June, 2006. Bram Cohen, “Incentives Build Robustness in BitTorrent,” in Proc. First Workshop on Economics of Peer-to-Peer Systems, Berkeley, June, 2003. H. Shojania and B. Li, “Parallelized Progressive Network Coding With Hardware Acceleration,” in Proc. IEEE International Workshop on Quality of Service (IWQoS) 2007, June, 2007.

In future work, it would be interesting to investigate the impact on system performance when the segments are not equal in size. ACKNOWLEDGMENT This work was supported in part by NSFC under Grant No. 60702054, 863 program of China under Grant No. 2006AA01Z203, and Shanghai Municipal R&D Foundation under Grant No. 07dz15004-1.

5608 Authorized licensed use limited to: Xiamen University. Downloaded on June 2, 2009 at 10:08 from IEEE Xplore. Restrictions apply.