UNI-MB - logo
UMNIK - logo
 
E-viri
Celotno besedilo
  • Tangsatjatham, Pittayut; Nupairoj, Natawut

    2016 13th International Joint Conference on Computer Science and Software Engineering (JCSSE), 2016-July
    Conference Proceeding

    Log processing can be very challenging, especially for environments with lots of servers. In these environments, log data is large, coming at high-speed, and have various formats, the classic case of big data problem. This makes anomaly detection very difficult due to the fact that to get good accuracy, large amount of data must be processed in real-time. To solve this problem, this paper proposes a hybrid architecture for log anomaly detection using Apache Spark for data processing and Apache Flume for data collecting. To demonstrate the capabilities of our proposed solution, we implement a SARIMA-based anomaly detection as a case study. The experimental results clearly indicated that our proposed architecture can support log processing in large-scale environment effectively.