nips nips2006 nips2006-96 nips2006-96-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Ling Huang, Xuanlong Nguyen, Minos Garofalakis, Michael I. Jordan, Anthony Joseph, Nina Taft
Abstract: We consider the problem of network anomaly detection in large distributed systems. In this setting, Principal Component Analysis (PCA) has been proposed as a method for discovering anomalies by continuously tracking the projection of the data onto a residual subspace. This method was shown to work well empirically in highly aggregated networks, that is, those with a limited number of large nodes and at coarse time scales. This approach, however, has scalability limitations. To overcome these limitations, we develop a PCA-based anomaly detector in which adaptive local data filters send to a coordinator just enough data to enable accurate global detection. Our method is based on a stochastic matrix perturbation analysis that characterizes the tradeoff between the accuracy of anomaly detection and the amount of data communicated over the network.
[1] BAI , Z.-J., C HAN , R. AND L UK , F. Principal component analysis for distributed data sets with updating. In Proceedings of International workshop on Advanced Parallel Processing Technologies (APPT), 2005.
[2] D REGER , H., F ELDMANN , A., PAXSON , V. AND S OMMER , R. Operational experiences with highvolume network intrusion detection. In Proceedings of ACM Conference on Computer and Communications Security (CCS), 2004.
[3] H UANG , L., N GUYEN , X., G AROFALAKIS , M., J ORDAN , M., J OSEPH , A. AND TAFT, N. In-network PCA and anomaly detection. Technical Report No. UCB/EECS-2007-10, EECS Department, UC Berkeley.
[4] JACKSON , J. E. AND M UDHOLKAR , G. S. Control procedures for residuals associated with principal component analysis. In Technometrics, 21(3):341-349, 1979.
[5] J ENSEN , D. R. AND S OLOMON , H. A Gaussian approximation for the distribution of definite quadratic forms. In Journal of the American Statistical Association, 67(340):898-902, 1972.
[6] K ERALAPURA , R., C ORMODE , G. AND R AMAMIRTHAM , J. Communication-efficient distributed monitoring of thresholded counts. In Proceedings of ACM International Conference on Management of Data (SIGMOD), 2006.
[7] K REIDL , P. O., W ILLSKY, A. Inference with minimal communication: A decision-theoretic variational approach. In Proceedings of Neural Information Processing Systems (NIPS), 2006.
[8] L AKHINA , A., C ROVELLA , M. AND D IOT, C. Diagnosing network-wide traffic anomalies. In Proceedings of ACM Conference of the Special Interest Group on Data Communication (SIGCOMM), 2004.
[9] L AKHINA , A., PAPAGIANNAKI , K., C ROVELLA , M., D IOT, C., KOLACZYK , E. D. AND TAFT, N. Structural analysis of network traffic flows. In Proceedings of International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS), 2004.
[10] L EVCHENKO , K., PATURI , R. AND VARGHESE , G. On the difficulty of scalably detecting network attacks. In Proceedings of ACM Conference on Computer and Communications Security (CCS), 2004.
[11] N GUYEN , X., WAINWRIGHT, M. AND J ORDAN , M. Nonparametric decentralized detection using kernel methods. In IEEE Transactions on Signal Processing, 53(11):4053-4066, 2005.
[12] PADMANABHAN , V. N., R AMABHADRAN , S., AND PADHYE , J. Netprofiler: Profiling wide-area networks using peer cooperation. In Proceedings of International Workshop on Peer-to-Peer Systems, 2005.
[13] P REDD , J.B., K ULKARNI , S.B., AND P OOR , H.V. Distributed learning in wireless sensor networks. In IEEE Signal Processing Magazine, 23(4):56-69, 2006.
[14] Q U , Y., O STROUCHOVZ , G., S AMATOVAZ , N AND G EIST, A. Principal component analysis for dimension reduction in massive distributed data sets. In Proceedings of IEEE International Conference on Data Mining (ICDM), 2002.
[15] S TEWART, G. W., AND S UN , J.-G. Matrix Perturbation Theory. Academic Press, 1990.
[16] Y EGNESWARAN , V., BARFORD , P., AND J HA , S. Global intrusion detection in the domino overlay system. In Proceedings of Network and Distributed System Security Symposium (NDSS), 2004.
[17] Z HANG , Y., G E , Z.-H., G REENBERG , A., AND ROUGHAN , M. Network anomography. In Proceedings of Internet Measurement Conference (IMC), 2005.