Scalable Content Delivery

Distributed On Demand Video Content Distribution

We propose, analyze, and implement a general architecture for massively parallel VoD content distribution. We allow for devices that have a wide range of reliability, storage and bandwidth constraints. Each device can act as a cache for other devices and can also communicate with a central server. Some devices may be dedicated caches with no co-located users. Our goal is to allow each user device to be able to stream any movie from a large catalog, while minimizing the load of the central server.

First, we architect and formulate a static optimization problem that accounts for various network bandwidth and storage capacity constraints, as well as the maximum number of network connections for each device. Not surprisingly this formulation is NP-hard. We then use coding and the Markov approximation technique in a primal-dual framework to devise a highly distributed algorithm which is provably close to the optimal. Next we test the practical effectiveness of the distributed algorithm in several ways. We demonstrate remarkable robustness to system scale and changes in demand, user churn, network failure and node failures via a packet level simulation of the system. Finally, we test our system with numerous experiments on a full implementation of the system on Amazon EC2 instances.

1. K. Lee, L. Yan, A. Parkeh and K. Ramchandran, “A VoD System for Massively Scaled, Heterogeneous Environments: Design and Implementation”, IEEE 21st International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2013), San Francisco, CA, August, 2013. Best Paper Finalist.

2. K. Lee, H. Zhang, Z. Shao, M. Chen, A. Parekh and K. Ramchandran, “An Optimized Distributed Video-on-Demand Streaming System: Theory and Design”*, The 50th Allerton Conference on Communication, Control and Computing, Monticello, IL, October, 2012. (invited paper)

3. S. Pawar, S. Rouayheb, H. Zhang. K. Lee and K. Ramchandran, “Codes for a Distributed Caching based Video-On-Demand System”, Asilomar Conference on Signals, Systems, and Computers, Pacific grove, CA, November, 2011.

4. Hao Zhang, Ziyu Shao, Minghua Chen, and Kannan Ramchandran, “Optimal Neighbor Selection in BitTorrent-like Peer-to-Peer Networks”, in Proceedings of ACM SIGMETRICS, San Jose, CA, US, June 7-11, 2011. (poster paper)

5. Hao Zhang, Minghua Chen and Kannan Ramchandran, “Scaling P2P Content Delivery Systems Reliably by Exploiting Unreliable System Resources,” IEEE MMTC E-Letter of December, 2009. (invited paper)

6. Hao Zhang, Jiajun Wang, Minghua Chen and Kannan Ramchandran, “Scaling Peer-to-Peer Video-on-Demand Systems Using Helpers,” IEEE International Conference on Image Processing (ICIP), Nov, 2009.

7. Hao Zhang and Kannan Ramchandran, “A Reliable Decentralized Peer-to- Peer Video-on-Demand System Using Helpers,” Picture Coding Symposium (PCS), May, 2009. (invited paper)