SaND: Sampling the Internet for Effective Network Anomaly Detection (2006-10)
Accurate traffic measurement is essential for both network management tasks (e.g.,capacity planning, traffic engineering) and security forensics (e.g., detecting malicious traffic such as portscans or DoS attacks). Various sampling techniques have been proposed for traffic measurements in high-speed backbone networks to reduce storage and processing overhead. There are well-studied trade-offs between the accuracy, efficiency, and scalability in choosing a specific sampling method or rate for traffic engineering purposes. However, a detailed study of whether sampled data is sufficient for anomaly detection is necessary.
This project seeks to answer this question: Does sampled data capture sufficient information for effective anomaly detection? Through experiments using real traffic traces, we have quantified how existing sampling schemes (e.g., random packet sampling, random flow sampling, smart sampling, and sample-and-hold) affect the performance of a waveletbased volume anomaly detection method and two portscan detection algorithms.
We believe that the lessons learned in our preliminary explorations can be leveraged to address these accuracy and efficiency tradeoffs in designing better sampling techniques. Anomaly detection operates on a significantly different information region, which is often overlooked by existing traffic accounting methods that target heavy-hitters. Better measurement techniques need to adapt to the needs of anomaly detections as well as traffic engineering, whether it be flow statistics or access patterns.
People
Faculty
Graduate Students and Alumni
- G. Huang, ECE
- J. Mai, ECE (PhD, 2008)
- L. Yuan, ECE (PhD, 2008)
Collaborators
- J. Xu, Georgia Institute of Technology
- A. Sridharan, Sprint ATL
- H. Zang, Sprint ATL
- T. Ye, Sprint ATL
Publications
G. Huang, A. Lall, C-N. Chuah, and J. Xu, "Uncovering Global Icebergs in Distributed Streams: Results and Implications," to appear in Journal of Network and Systems Management. [pdf]
J. Mai, A. Sridharan, H. Zang, and C-N. Chuah, "Fast Filtered Sampling: Catching Mice and Elephants with One Net," Elsevier Computer Networks, vol. 54, no. 11, pp. 1885-1898, August 2010. [pdf]
R. Keralapura, A. Nucci, and C-N. Chuah, "A Novel Self-Learning Architecture for P2P Traffic Classification in High Speed Networks," Elsevier Computer Networks, vol. 54, no. 7, pp. 1055-68, May 2010. [pdf]
S. Raza, G. Y. Huang, C-N. Chuah, S. Seetharaman, and J. P. Singh, "MeasuRouting: A Framework for Routing-Assisted Traffic Monitoring," IEEE INFOCOM, March 2010. [pdf]
R. Keralapura, A. Nucci, and C-N. Chuah, "Self-Learning Peer-to-Peer Traffic Classifier," IEEE Conference on Computer Communications and Networks (ICCCN), August 2009. [pdf]
G. Huang, A. Lall, C-N. Chuah, and J. Xu, "Uncovering Global Icebergs in Distributed Monitors," IEEE IWQoS, July 2009. [pdf]
F. Khan, L. Yuan, C-N. Chuah, and S. Ghiasi, "Programmable and Real-time Network Traffic Measurements," ACM/IEEE Symposium on Architectures for Networking and Communications Systems, November 2008. [pdf]
J. Mai, L. Yuan, and C-N. Chuah, "Detecting BGP Anomalies with Wavelet," IEEE/IFIP Network Operations and Management Symposium (NOMS), April 2008. [pdf]
L. Yuan, C-N. Chuah, and P. Mohapatra, "ProgME: Towards Programmable Network MEasurement," ACM SIGCOMM, August 2007. [pdf]
J. Mai A. Sridharan, C-N. Chuah, T. Ye, and H. Zang, "Impact of Packet Sampling on Portscan Anomaly Detection," IEEE Journal on Selected Areas of Communications - Special Issue on Sampling the Internet, vol. 24, no. 12, pp. 2285-2298, December 2006. [pdf]
J. Mai, C-N. Chuah, A. Sridharan, T. Ye, and H. Zang, "Is Sampled Data Sufficient for Anomaly Detection?" ACM/USENIX Internet Measurement Conference, October 2006. [pdf]
Talks
"Impact of Sampling on Anomaly Detection," DIMACS/DyDAn Workshop on Internet Tomography, Rutgers University, NJ, May 2008. [pdf]
Funding
This project is supported by National Science Foundation CyberTrust Grant #0716831 and UC Micro program with matching funds from Sprint and Narus.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.